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ABSTRACT 



One of the problems common to efforts to set skill standards 
for various occupations is deciding on the facility needed for dealing with 
the printed word. A similar problem faces those trying to define "workplace 
literacy.” This workbook brings together resources to make them available to 
those who are trying to deal with this education issue. The following 
materials are included: (1) "Becoming Literate about Literacy" (Paul E. 

Barton) ; (2) "Print Use Profiles" (Judith Shaul Norback) ; (3) "Literacy 

Levels for 90 Occupations" (Paul E. Barton); (4) "Reading and Writing Skill 
Needs in 22 Industries" (Joan Wills) ; (5) "Elements of Job Literacy" (Judith 

Shaul Norback); (6) "Different Approaches to Establishing Levels, Scales, and 
Equivalencies" (Judith Shaul Norback); (7) "Workplace Competencies: The Need 
To Improve Literacy and Employment Readiness" (Paul E. Barton and Irwin S. 
Kirsch) ; (8) "Setting Performance Standards: Content, Goals, and Individual 

Differences" (Bert F. Green) ; and (9) "Literacy: Economic Key for the New 
Millennium" (Richard J. Coley). Most individual articles contain references. 
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Introduction 



One of the problems common to all efforts to set skill standards for 
various occupations is deciding on the facility needed for dealing with the 
printed word. A similar problem confronts those trying to define broader 
"workplace literacy" requirements that are common to all occupations. 

We have developed the "workbook" to bring together resources that are 
scattered around and make them available to people around the country who 
are trying to deal with this important education issue. The materials provided 
in this volume are described below: 

Becoming Literate About Literacy, by Paul E. Barton 

Originally published by the Policy Information Center in 1994, it was 
designed to be a primer on what was learned from the 1992 National Adult 
Literacy Study (NALS). It provides the definition of the prose, document, and 
quantitative literacy scales and describes the proficiency of the adult 
population. It provides examples of tasks that adults at different levels of 
literacy are able to perform. It can be used to understand the references to the 
National Adult Literacy Study (NALS) in the Chapters below. 

Print Use Profiles, by Judith Shaul Norback of the Center for Skills Enhancement 
and Garlie Forehand of Educational Testing Service. 

This was written for the Workbook and has not been previously 
published. Judith Shaul Norback and colleagues have carried out Job Literacy 
Analyses (JLA) for 29 occupations. They provide detailed examinations of the 
print materials used on the job. Norback and Forehand have also translated the 
results of six of these analyses into the five levels of literacy used in the 



National Adult Literacy Study. JLA is the most in-depth approach available to 
establish print-use requirements, and translating this analysis into the NALS 
levels permits linking to the data base that provides information on the entire 
adult population. 

Literacy Levels for 90 Occupations, by Paul E. Barton. 

One way to look at the literacy requirements of occupations is to assess 
the literacy levels of people working in those occupations. The National Adult 
Literacy Assessment Study (NALS) permits this for occupations in which the 
sample size is large enough. This article provides assessment results for about 
90 occupations, showing the band (confidence interval) on the Prose Literacy 
Scale in which the true mean scale score lies. It also provides data for document 
and quantitative literacy from which to calculate these confidence intervals. 
While these are the actual literacy levels of real people working in real jobs, we 
do not know whether their literacy skills are above or below what the jobs 
actually require. 

Reading and Writing Skill Needs in 22 Industries , by Joan Wills, Center for 
Workforce Development. 

This is taken from her 1996 publication. In Search of Commonalities, in 
which she extracts knowledge and skill requirements from the 22 skill 
standards projects funded by the Federal Government. For example, you can 
find here the reading skill requirements for an entry-level welder. 
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Elements of Job Literacy (The Norback Job Literacy Structure), by Judith Shaul 
Norback, Center for Skills Enhancement, 1996. 

Norback has identified 26 categories of skills that constitute job literacy, 
of which 18 are within the areas of quantitative, document, and prose literacy. 
Others include such skills as following directions, computer use, and basic 
communication. For each of these 26 categories, she specifies the elements that 
constitute the category. This can serve as a checklist for looking at the 
requirements of any job or occupation. 

Different Approaches to Establishing Levels, Scales, and Equivalencies, by Judith 
Shaul Norback, Center for Skills Enhancement (prepared for the Center for 
Workforce Development and the National Skill Standards Board). 

A first step in developing skill standards is to identify essential skills 
and knowledge for specific occupations. To communicate this to others, to 
determine the extent of skills and knowledge required, and to determine the 
degree to which individuals possess them, it is necessary to define levels 
and/ or create scales. In this paper, Norback gives examples of how this is done 
in the U.S. and in two other countries. 

Workplace Competencies : The Need to Improve Literacy and Employment 
Readiness, by Paul E. Barton and Irwin S. Kirsch, 1990. 

This is a general discussion of literacy, the labor market, and 
occupations, with recommendations for public policy. It was published by the 
U.S. Department of Education, with a Foreword by Christopher Cross. 
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Setting Performance Standards: Content, Goals, and Individual Differences, by 
Bert F. Green, John Hopkins University. 

After setting content standards," describing what people in various 
occupations must know and be able to do, and after expressing these in scales 
or levels, it is necessary to set "performance standards." Performance standards 
tell us how much you need to know or be able to do. In this paper published by 
the ETS Policy Information Center, Bert Green describes the approaches that are 
available for doing this. 

Literacy: Economic Key for the New Millennium, by Richard J. Coley, ETS Policy 
Notes, 1996. 

This publication is drawn from Economy and Society: Results of the First 
International Adult Literacy Study, published by the Organization for Economic 
Cooperation and Development, 1996. A key section is an international 
comparison of prose, document, and quantitative literacy by major 
occupational categories. 
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li*ter (lector), n. a unit of capacity redefined in 1964 by 
a reduction of 28 parts in a million to be exactly equal to 
one cubic decimeter. It is equivalent to 1.0567 U.S. liquid 
quarts and is equal to the volume of one kilogram of dis- 
tilled water at 4 5 C. Abbr.: 1 Also, esp. Brit., litre. [1800- 
10; < F Litre, back formation from litron an old measure 
of capacity, deriv. (with - on n. suffix) of ML litra < Gk 
litra pound] 

!it*er«a*cy (lit / er o se), n. 1. the quality or state of 
being literate, esp. the ability to read and write. 2 . pos- 
session of education: to question someone’s literacy. 3. a 
person’s knowledge of a particular subject or field: to ac- 
quire computer literacy. [1880-85; liter(ate) -f -acy] 
Syn. 2. learning, culture. 

literacy an examination to determine whether 

a person meets the literacy requirements for voting, 
serving in the armed forces, etc.; a test of one’s ability to 
read and write. 11865-701 
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Summary and Highlights 



In 1992, under contract 
to the National Center 
for Education Statistics, 
Educational Testing 
Service assessed the 
literacy proficiencies of a 
nationally representative 
sample of all adults, age 
16 and over. That assess- 
ment, the National Adult 
Literacy Survey, pro- 
duced a profile of the 
entire adult population’s 
English literacy skills 
along three proficiency 
scales, each ranging 
from 0 to 500: prose, 
document, and quantita- 
tive. The survey em- 
ployed the following 
definition of literacy: 

Using printed and 
written information to 
function in society, to 
achieve one’s goals, 
and to develop one’s 
knowledge and 
potential. 

Under this definition, 
literacy skills are on a 
continuum, and there is 
no single point that 
separates “illiterates” 
from those who are 
literate. Accordingly, 
each of the three scales 
(corresponding to the 
three types of literacy: 
prose, document, and 
quantitative) were 
divided into five pro- 
gressive levels that are 
characterized by tasks of 
increasing complexity. 

For example, on the 
prose scale, tasks in 
Level 1 (0 to 225) 
include locating or 
identifying information 
in a brief text, whereas 
tasks in Level 4 (326- 




375) include making 
inferences and synthesiz- 
ing information from 
complex and lengthy 
texts. 

This volume draws on 
the results from the first 
official report of the 
National Adult Literacy 
Survey, issued in Septem- 
ber 1993. The report, 
titled Adult Literacy in 
America, was written by 
Irwin Kirsch, Ann 
Jungeblut, Lynn Jenkins, 
and Andrew Kolstad. 

Prose Literacy 

• One in five adults 
performed at the lowest 
Prose level (Level 1). 

They can locate a single 
piece of information in a 
text, when there is little 
distracting information to 
deal with (page 6).* 

• About one in four 
adults are at Prose Level 2. 
They can locate a single 
piece of information, 
when there is distracting 
information, and they 
can integrate, compare, 
and contrast information 
(page 7). 

• Three in 10 adults are 
at Level 3. They can 
match information in a 
text to that in the direc- 
tive given to them, when 
some inference is required 
(page 8). 

• One in six adults are at 
Level 4. They can inte- 
grate and synthesize 
information from com- 
plex or lengthy passages 
and make more complex 
inferences (page 9). 



• Just one in 33 adults — 
at Level 5 — can make 
high-level inferences 
based on text and can 
find information in dense 
text with considerable 
distracting information 
that might seem plau- 
sible but is incorrect 
(page 10). 



Document Literacy 

• Almost one in four 
adults are at Level 1 in 
Document Literacy. They 
can match information 
and fill in personal 
information on forms 
(page 11).* 

• Almost three in 10 are 
at Level 2. They can 
match information in 
documents that contain 
distractors, and they can 
integrate information 
from several places in a 
document (page 12). 

• Three in 10 adults are 
at Level 3. They can 
integrate several pieces 
of information and deal 
with rather complex 
tables and graphs that 
contain information not 
pertaining to the task 
(page 13). 

• Almost one in seven 
adults are at Level 4. 

They can do tasks 
requiring greater infer- 
ences and involving 
more detailed informa- 
tion (page 14). 

• One in 33 adults are at 
Level 5. They can use 
complex documents 
that contain distracting 

-- 10 



information and make 
high-level inferences 
(page 15). 

Quantitative Literacy 

• Over one in five 
adults are at Level 1. 
They can perform single, 
relatively simple opera- 
tions (page 16).* 

• One in four adults are 
at Level 2. They can 
perform an arithmetic 
operation using numbers 
given to them or easily 
located in the material 
(page 17). 

• Three in 10 of all 
adults are at Level 3. 
They can solve prob- 
lems with two or more 
numbers that must be 
found in printed material 
(page 18). 

• One in six adults. are 
at Level 4 and can 
handle two or more 
arithmetic operations in 
sequence (page 19). 

• One in 25 adults 
perform at Level 5. They 
can perform multiple 
arithmetic operations 
sequentially (page 20). 



* Level 1 is from 0 to 225; some 
adults are at the lower end of 
this level where they cannot 
perform even these routine 
tasks. 
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Introduction 



■ Among 40 million 
adults in the lowest level 
of Prose Literacy (Level 1), 
one in four have some 
kind of limiting physical 
or mental condition that 
may impair their literacy 
skills, or make improve- 
ment more difficult 
(page 21). 

■ Literacy and the 
exercise of citizenship in 
a democracy are related. 
Higher levels of literacy 
go hand-in-hand with 
voting or obtaining 
information from news- 
papers and magazines 
(page 22). 

■ The prison population 
is considerably less 
literate than the general 
population. Almost seven 
in 10 prisoners are at 
Levels 1 and 2 of Prose 
Literacy, compared with 
less than half of the 
general population 
(page 23). 

■ Prose proficiency rises 
with education level. 
Three-fourths of adults 
with between zero and 
eight years of school are 
in Level 1, as are four in 
10 high school dropouts. 
Over 75 percent of two- 
year college graduates, 
and 85 percent of four- 
year college graduates, 
reach at least Level 3. 
However, just 4 and 10 
percent, respectively, 
reach Level 5 (page 24). 

■ While fewer middle- 
age adults are at lower 
literacy levels than 
adults, older 

are considerably 

4 




less literate. However, 
older adults also have 
less education than 
middle-age or young 
adults (page 27). 

■ Parents’ education 
gready influences the 
literacy proficiency levels 
reached by their chil- 
dren. Even among 
respondents with similar 
levels of education, 
those with better 
educated parents have 
higher prose literacy 
proficiencies (page 28). 

■ Poverty and literacy 
are closely related; 

43 percent of adults in 
Level 1 are poor or near 
poor, compared with just 
4 percent in Level 5 
(page 25). 

■ Weekly wages and 
weeks worked per year 
rise with literacy level: 
Adults at Level 5 earn 

2 1/2 times as much as 
adults at Level 1, and 
work more than twice as 
many weeks (page 26). 

■ The average prose 
proficiencies of Black, 
Asian, Hispanic/Puerto 
Rican, Hispanic/Cuban, 
Hispanic/Central and 
South American, and 
Hispanic/Mexican adults 
are lower than the 
average proficiency of all 
laborers (page 29). 



Our nation’s growing 
concern about its state of 
literacy has been evident 
in local and state legisla- 
tion, programs at all 
levels of government, 
and in a multitude of 
campaigns urging action. 
We are fortunate that the 
concern and the efforts 
to increase literacy exist, 
but unfortunately, these 
campaigns have had 
very little to go on in 
terms of hard informa- 
tion. There has been a 
lack of reliable informa- 
tion about the state of 
literacy in the population 
at large. 

True, numbers have 
been used in abundance — 
most of us have seen at 
least one of the adver- 
tisements stating some- 
thing to the effect of “x 
number of Americans 
can’t read this ad” or “x 
number of high school 
graduates cannot read 
their high school diplo- 
mas.” While the adver- 
tisements may serve as 
a wake-up call to the 
public, they are, none- 
theless, misrepresenta- 
tions of a complex ' 
problem. These numbers 
are based on distorted 
information and, by 
oversimplification, can 
hinder efforts to improve 
literacy. 

One consequence of 
presenting the literacy 
problem in simple terms 
of how many people 
can’t read or write is that 
it fosters a widespread 
belief that assessing 
literacy is a simple 
process — it can all be 
boiled down to, a single 



number. As we proceed 
to examine the results of 
the first, large-scale 
national literacy assess- 
ment of all adults (age 16 
and over), the first step 
we must take is to dispel 
that myth and become 
literate about literacy. We 
owe it to ourselves and 
to the thousands of 
people who are dedi- 
cated to improving 
literacy in this nation. 

This summary report 
draws upon the results of 
the National Adult 
Literacy Survey, man- 
dated by Congress and 
carried out by the Na- 
tional Center for Educa- 
tion Statistics through a 
contract with Educational 
Testing Service. Adult 
Literacy in America is 
the first report from the 
survey, written by Irwin 
S. Kirsch, Ann Jungeblut, 
Lynn Jenkins, and 
Andrew Kolstad. 

The term literacy is 
often used as just the 
opposite of illiteracy, and 
that term has come to be 
used as meaning that a 
person cannot read at all, 
cannot decode the 
printed word, and does 
not comprehend what is 
written. But literacy has a 
much richer and deeper 
meaning than that. Its 
dictionary definitions 
range from being able to 
read and write to being a 
well informed, educated 
person and to being 
familiar with literature. 

In the young adult 
literacy assessment of 
1985, in an assessment of 
populations served by 
the U.S. Department of 



Labor in 1990, and in this 
new assessment of all 
adults, the same defini- 
tion was adopted by a 
broadly representative 
group of experts, as 
follows: 

Using printed and 
written information to 
function in society, to 
achieve one ’s goals, 
and to develop one’s 
knowledge and 
potential. 

This definition encom- 
passes the multifaceted 
nature of literacy — it is 
not a single skill, i.e. 
reading, but rather 
literacy involves an 
ordered set of skills that 
are necessary to accom- 
plish various tasks in 
various contexts. Balanc- 
ing a checkbook, obtain- 
ing information from a 
train schedule, and 
understanding a passage 
from a novel — each of 
these tasks call for 
literacy skills but not the 
same skills. The National 
Adult Literacy Survey 
employed three profi- 
ciency scales to measure 
these distinct skills: 
prose, document, and 
quantitative. Their defini- 
tions are as follows: 

Prose literacy — the 
knowledge and skills 
needed to understand 
and use information from 
texts that include editori- 
als, news stories, poems, 
and fiction; for example, 
finding a piece of infor- 
mation in a newspaper 
article, interpreting 
instructions from a 




warranty, inferring a 
theme from a poem, 
or contrasting views 
expressed in an editorial. 

Document literacy — 
the knowledge and skills 
required to locate and 
use information contained 
in materials that include 
job applications, payroll 
forms, transportation 
schedules, maps, tables, 
and graphs; for example, 
locating a particular 
intersection on a street 
map, using a schedule to 
choose the appropriate 
bus, or entering informa- 
tion on an application 
form. 

Quantitative literacy — 
the knowledge and skills 
required to apply arith- 
metic operations, either 
alone or sequentially, 
using numbers embed- 
ded in printed materials; 
for example, balancing a 
checkbook, figuring out 
a tip, completing an order 
form, or determining the 
amount of interest from a 
loan advertisement. 

More than 400 trained 
interviewers conducted 
the survey, administering 
the literacy assessment in 
nearly 27,000 homes 
during the first eight 
months of 1992. A 
separate assessment was 
carried out for the prison 
population. The assess- 
ment consisted of a set of 
tasks that simulated real 
life situations encoun- 
tered at home, at work, 
and in the community. 
Almost all the tasks 
required participants to 

; X* 



construct their responses, 
as opposed to choosing 
a response from multiple 
choices. 

To ensure the broadest 
possible coverage of 
prose, document, and 
quantitative literacy, 
there were a total of 166 
tasks requiring a variety 
of information-process- 
ing skills and strategies. 
For purposes of effi- 
ciency, each survey 
participant responded to 
a subset of these tasks, 
which were compiled in 
booklets that took about 
45 minutes to complete. 
Additionally, participants 
spent about 20 minutes 
completing personal 
background question- 
naires. The background 
data informs our under- 
standing of the ways in 
which personal character- 
istics (race/ethnicity, age, 
educational attainment, 
labor market experiences, 
access to printed materi- 
als, etc.) are associated 
with demonstrated perfor- 
mance on each literacy 
scale. 

One way of presenting 
the results would be to 
describe how all adults 
did on all 166 tasks. To 
do so would place a very 
large burden on the 
reader, making it difficult 
to conclude what the 
state of literacy is for 
each population group. 

To help make such 
judgments, statistical 
methods were used to 
construct proficiency 
scales from the 
answers to the tasks, 
scales similiar to the 
ones used to report the 



results of the SAT. The • 
National Adult Literacy 
Survey scales range from 
0 to 500. Individual tasks 
are assigned scores along 
the scales, to give 
examples of what people 
with a particular score 
are likely to be able to 
do. The scales were then 
divided into five levels, 
each encompassing a 
defined score range, 
such as from 226 to 275. 
Individuals scoring 
within these scale levels 
would have a high 
probability of performing 
the tasks at that level 
successfully (a high 
probability is defined as 
at least 80 percent of the 
time).* 

The sections that 
follow present highlights 
of the adult literacy 
survey. The first parts 
include data on each of 
the five levels of prose, 
document, and quantita- 
tive literacy. For each 
level, we describe what a 
typical adult at that 
literacy level can do, and 
also show an example of 
an item or exercise 
actually used in the 
assessment. We also 
show the percentage of 
respondents that reach 
that particular level for 
each racial/ethnic group. 
The following sections 
show the relationship of 
literacy to respondents’ 
social and economic 
characteristics. The final 
section offers some 
concluding thoughts. 



* And they would have a small 
chance of performing tasks at 
a higher level. 
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PROSE LITERACY, LEVEL 1 (0 TO 225) 



One in five adults 
performed at the 
lowest Prose level 
(Level 1). They can 
locate a single piece 
of information in a 
text, when there is 
little distracting infor- 
mation to deal with. 
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read relatively short text to locate a single piece of 
information that is identical to, or synonymous with, 
the information given in the question or directive. 
Distracting information is minimal.* 

They are Ukely to succeed at tasks such as: 

• Identify the country in a short article ( 149) 

• Locate one piece of information in a sports article 
(210) (See below) 

• Underline a sentence explaining the action stated 
in a short article (225) 



However, some were at the lower end of this level where they 
could not perform even these routine tasks. 



EXAMPLE TASK 

Underline the sentence that tells wbat Ms. Cbanin ate during the swim 



ERIC 



Swimmer completes 

Manhattan marathon 



The Associated Press 
NEW YORK — University of Maryland 
senior Stacy Chanin on Wednesday became 
the first person to swim three 28-mile laps 
around Manhattan. 

Chanin, 23, of Virginia, climbed out of 
the East River at 96th Street at 9:30 p.m. 
She began the swim at noon on Tuesday. 

A spokesman for the swimmer, Roy 
Brunett, said Chanin had kept up her 
strength with "banana and honey" 
sandwiches, hot chocolate, lots of water 
and granola bars." 



Chanin has twice circled Manhattan 
before and trained for the new feat by 
swimming about 28.4 miles a week. The 
Yonkers native has competed as a swimmer 
since she was 15 and hoped to persuade 
Olympic authorities to add a long-distance 
swimming event. 

The Leukemia Society of America 
solicited pledges for each mile she swam. 

In July 1983, Julie Ridge became the 
first person to swim around Manhattan 
twice. With her three laps, Chanin came 
up just short of Diana Nyads distance 
record, set on a Florida-to-Cuba swim. 
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PROSE LITERACY, LEVEL 2 (226 TO 275) 



About one in four 
adults are at Prose 
Level 2. They can 
locate a single piece 
of information when 
there is distracting 
information, and they 
can integrate, com- 
pare, and contrast 
information. 




They can: 

locate a single piece of information when there is 
distracting information or other information that 
seems plausible as the answer but is incorrect. They 
can also integrate, compare, or contrast two or more 
pieces of information. 

They are likely to succeed at tasks such as: 

• Underline meaning of a term given in government 
brochure on supplemental security income (226) 

• Locate two features of information in a sports 
article (250) 

• Interpret instructions from an appliance warranty 
(275) (See below) 



EXAMPLE TASK 



A manufacturing company provides its customers with the fol- 
lowing instructions for returning appliances for service: 



When returning appliance for servicing, include a note telling as clearly and 
as specifically as possible what is wrong with the appliance. 



A repair person for the company receives four appliances with the 
following notes attached. Circle the letter next to the note which 
best follows the instructions supplied by the company. 



C 



D 




This radio is broken. Please 
repair and return by United 
Parcel Service to the address on 
my slip. 



The clock does not run 
correctly on this clock 
radio. I tried fixing it, but 
I couldn't. 



My clock radio is not working. It 
B stopped working right after I 
used it for five days. 



The alarm on my clock 
radio doesn't go off at the 
time I set. It rings 15-30 
minutes later. 
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PROSE LITERACY, LEVEL 3 (276 TO 325) 



EXAMPLE TASK 



Three in 10 adults can 
match information in 
a text to that in the 
directive given to 
them, when some 
inference is required. 




They can: 

match information in the 
text and in the task when 
low level inferences are 
required, integrate 
information from dense 
or lengthy text, and 
generate a response 
based on information 
easily identified in the 
text. 

They are likely to 
succeed at tasks 
such as: 

• Write a brief letter 
explaining error made 
on a credit card bill 

( 288 ) 

• Read a news article 
and identify a sen- 
tence that provides 
interpretation of a 
situation (304) 

• Read a lengthy article 
to identify behaviors 
that meet a stated 
condition (310) 




List two things that Chen became involved in or has done to help resolve 
conflicts due to discrimination. 



IDA CHEN is the first Asian- American woman to 
become a judge of the Commonwealth of Pennsylv ania . 



She understands 
discrimination because she 
has experienced it herself. 

Soft-spoken and eminently dignified, 
Judge Ida Chen prefers hearing about a 
new acquaintance rather than talking 
about herself. She wants to know about 
career plans, hopes, dreams, fears. She 
gives unsolicited advice as well as 
encouragement. She instills confidence. 

Hex father once hoped that she 
would become a professor. And she 
would have also made an outstanding 
social worker or guidance counselor. 

The truth is that Chen wears the caps of 
all these professions as a Family Court 
judge of the Court of Common Pleas of 
Philadelphia County, as a participant in 
public advocacy for minorities, and as a 
particularly sensitive, caring person. 

She understands discrimination 
because she has experienced it herself. 
As an elementary school student, Chen 
tried to join the local Brownie troop. 
"You can't be a member," she was told. 
"Only American girls are in the 
Brownies." 

Originally intent upon a career as a 
journalist, she selected Temple Univer- 
sity because of its outstanding journal- 
ism department and affordable tuition. 
Independence being a personal need, she 
paid for her tuition by working for 
Temple's Department of Criminal 
Justice. There she had her first encoun- 
ter with the legal world and it turned 
her career plans in a new direction — 
law school. 

Through meticulous planning, Chen 
was able to earn her undergraduate 
degree in two and a half years and she 
continued to work three jobs. But when 
she began her first semester as a Temple 
law student in the fall of 1973, she was 
barely able to stay awake. Her teacher 
Lynne Abraham, now a Common Pleas 
Court judge herself, couldn't help but 
notice Chen yawning in the back of the 
class, and when she determined that 
this student was not a party animal but 
a workhorse, she arranged a teaching 
assistant's job for Chen on campus. 

After graduating from Temple Law 
School in 1976, Chen worked for the 
U.S. Equal Employment Opportunity 
Commission where she was a litigator 
on behalf of plaintiffs who experienced 
discrimination in the workplace, and 



then moved on to become the first 
Asian-American to serve on the 
Philadelphia Commission on Human 
Relations. 

Appointed by Mayor Wilson Goode, 
Chen worked with community leaders 
to resolve racial and ethnic tensions and 
also made time to contribute free legal 
counsel to a variety of activist groups. 

The "Help Wanted" section of the 
newspaper contained an entry that 
aroused Chen's curiosity — an ad for a 
judge's position. Her application 
resulted in her selection by a state 
judicial committee to fill a seat in the 
state court. And in July of 1988, she 
officially became a judge of the Court of 
Common Pleas. Running as both a 
Republican and Democratic candidate, 
her position was secured when she won 
her seat on the bench at last Novem- 
ber's election. 

At Family Court, Chen presides over 
criminal and civil cases which include 
adult sex crimes, domestic violence, 
juvenile delinquency, custody, divorce 
and support. Not a pretty picture. 

Chen recalls her first day as judge, 
hearing a juvenile dependency case — 

"It was a horrifying experience. I broke 
down because the cases were so 
depressing," she remembers. 

Outside of the courtroom, Chen has 
made a name for herself in resolving 
interracial conflicts, while glorying in 
her Chinese-American identity. In a 
1986 incident involving the desecration 
of Korean street signs in a Philadelphia 
neighborhood, Chen called for a 
meeting with the leaders of that 
community to help resolve the conflict. 

Chen's interest in community 
advocacy is not limited to Asian 
communities. She has been involved in 
Hispanic, Jewish and Black issues, and 
because of her participation in the 
Ethnic Affairs Committee of the Anti- 
Defamation League of B'nai B'rith, 

Chen was one of 10 women nationwide 
selected to take part in a mission to 
Israel. 

With her recently won mandate to 
judicate in the affairs of Pennsylvania's 
citizens, Chen has pledged to work 
tirelessly to defend the rights of its 
people and contribute to the improve- 
ment of human welfare. She would have 
made a fabulous Brownie. 

— Jessica Schultz 
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PROSE LITERACY, LEVEL 4 (326 TO 375) 



One in six adults 
can integrate and 
synthesize informa- 
tion from complex or 
lengthy passages and 
make more complex 
inferences. 




They can: 

match text with multiple features, integrate or 
synthesize information from complex or lengthy 
passages, and make more complex inferences. 



They are likely to succeed at tasks such as: 

* State in writing an argument made in a 
lengthy newspaper article (328) 

* Contrast views expressed in two editorials 

on technologies available to make fuel-efficient 
cars (359) (See below) 

* Compare two metaphors used in a poem (374) 



EXAMPLE TASK 



Contrast Dewey’s and Hanna’s views about the existence of technologies that can be used tn 
produce more fuel-efficient cars while maintaining the size of the cars 



Demand cars with 
better gas mileage 



Don’t demand end 
to cars people want 



By Robert Dewey 
Guett columnist 



WASHINGTON — Wcaiiif Auto- 
maker* ere resurrecting their heavy* 
BMtal dinosaur*, aka pi gustier*. 



oow-car mileage hae declined to 2 B 2 mike 

per gallon — the 1886 level Tb reverie 
thie trend. Congress must significantly 
increase existing fee-mileage standards. 

More than half our Nobel laureates 
and 700 members of the National Aead- 
emy of Sciences recently called global 
warming "the most serious environmen- 
tal threat of the 21st century.* In 1989, 
oil imports climbed to a near-record 46% 
of U.S. consumption. Increasing gas 
mileage is the single biggest step can 

take to reduce oil imports and curb global 

warming. Greeter efficiency also lowers 
outrode deficit (oil imports represent 
40% of it) and decreases the need to drill 
in pristine areas. 

Bigg er en gines and bigger ears mean 
biggm profits for automakers, who offer 
u* the products they want os to buy. 
More than aver, Americans want prod- 
ucts that have less of an environmental 
impact But with only a few fuel -efficient 
»rs to choose from, how do w* find ones 
that meet all our needs? 

Government studies show sutomakers 
have the technology to dramatically im- 



prove gas mileage — while mnintgitwrrg 
the 1987 levels of comfort, performance 
and sue mix of vehicles. Automakers al s o 
have the ability to their products 
safer. The cost of these im p rovements 
wfll be offset by savingi at the gas pump! 

Car* can average 45 napg and light 
trucks 35 tnpf primarily by ntili«n g an- 
gina and transmission technologies al- 
ready on a few cars today. Further im- 
provements are possible by using tech- 
nologies like the two-stroke engine and 
better aerodynamics that have been de- 
veloped but not used. 

When the current vehicle efficiency 
were proposed in 1974, Ford 
wrongly predicted that they "would re- 
quire either all eub-Pinto-sixed vehicles 
or soma mix of vehicles ranging from s 
sub-euboompect to perhaps a Maverick.* 
At that time. Congress required a 100% 
efficiency increase; raising ges mileag e 
to 46 mpf requires only a 60% increase. 

Americans want comfortable, safe and 
efficient care. If automakers wont pro- 
vide them. Congress must mandate them 
when it considers the issue this summer. 

Let's hope lawmakers put the best in- 
terest of the environment and the nation 
shead of the automakers' lobbyists end 
political action committees. 

Rot*rt Dwvry u a fa, O* Eju*- 

rommnual Aeum FommdaUom. 



By Thomas H, Hanna 
Guest col umni st 



DETROIT — Do Americana look for- 
ward to the day when thsyll have to haul 

groceries, shuttle the kids to end from 
■chool or take femily vacations in compact 
and subcom pact cars? 

I doubt it — which is why UA an d 
import carmakers oppose the 40-tmlee- 
P«-B»lloo to 45 mpg c o r porate average 
rael economy mandates that are 
pushing in Congrm, either to curb tailpipe 
cwtwn diaridp vnidoni favcPUM ofillMd 
global warming or for energy conservation. 

Since the mid-1970s, automakers have 
doubled the fleet average foal economy of 
new care U>28mpg— and further program 
win be 

Compact and subcompact can with 
mileage of 40 mpg or better are cow 
available, yet they appeal to only 5% of 
UA ear buyers. 

But to achieve a UA fleet average of 40 
mpg to 45 mpg. carmakers would have to 

sharply limit the availability of femily- 
eixa models and dwtmat^ny fa ^ a j ^ 
and weight of most car*. 

There simply are not magic technolo- 
gies to meet such a standard. 

Almost every car now sold in the USA 



would have to be drastically downsized, 
and many would be obsolete. 

Aa a result, Americ an a each 

be unable to buy the vehidoo i 

for their needs: mid- and famUy-eisa 
models, luxury automobile, mlm-vana, 

small trucks and utility v^dclaa. 

The fleet shift to compacts eobesan- 

pacts could alro form the doebw of assem- 
bly plants, supplier firom ami deakrahipB. 
at east of thousands of U A jobs. 

Although a growing number of a d an t ieta 
are skeptical of global warming; the Issue 
deserve# thorough international s cientific 

•yaluation, not premature unilateral U A 
action. 

Carbon dioxide emissions from U A ve- 
hicles total leas than 2.6% of worldwide 
*greenhou ac* gases. Even doubling todayfr 
corporate average Rad — *»"Tqy for U A 
ears — if technically possible— > would cut 
tboee gases about .6% 

Whatever the motivation — alleged 
global wanning or energy conservation — 
die stake# are high for milKon* of Amen- 
cans and thousands of UA jobs in unreal- 
istic corporate average fuel economy 
mandate#. 



TKomo4 H. HoMm u prukUmt mm/ tkmf in 

dfcwrtftkt Hour WhuLi Hmmafmsvtrm Ammrim 
tumtftktL’mmdSuumm 

tfOSA TUmy. 
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PROSE LITERACY, LEVEL 5 (376 TO 500) 



EXAMPLE TASK 



Just one in 33 adults 
can make high level 
inferences based on 
text and can find 
information in dense 
text with considerable 
distracting informa- 
tion that might seem 
plausible but is 
incorrect. 
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They can: 
find information in a 
dense text that contains 
considerable information 
that is distracting — 
information that may 
seem plausible as an 
answer but is incorrect. 
Also, they can make 
high-level inferences or 
use specialized back- 
ground knowledge. 



They are likely to 
succeed at tasks 
such as: 

• Compare the 
approaches stated 
in a narrative on 
growing up (382) 

• Summarize two ways 
lawyers may challenge 
prospective jurors 
(410) (See below) 

• Interpret a brief 
phrase from a lengthy 

^ news article (423) 

ERIC 



Identify and summarize the two kinds of challenges that attorneys use 
while selecting members of a jury. 



DO YOU HAVE A QUESTION? 



QUESTION: What is the new program for 
scheduling jurors? 

ANSWER: This is a new way of organizing 
and scheduling jurors that is being Intro- 
duced all over the country. The goals of 
this program are to save money. Increase 
the number of citizens who are summoned 
to serve and decrease the Inconvenience 
of serving. 

The program means that instead of call- 
ing jurors for two weeks, jurors now serve 
only one day, or for the length of one trial 
If they are selected to hear a case. Jurors 
who are not selected to hear a case are 
excused at the end of the day, and their 
obligations to serve as jurors are fuHHled 
for three years. The average trial lasts 
two days once testimony begins. 

An Important part of what is called the 
One Day - One Trial program is the 
"standby" juror. This is a person called to 
the Courthouse if the number of cases to 
be tried requires more jurors than origi- 
nally estimated. Once called to the Court- 
house, the standby becomes a "regular* 
juror, and his or her service is complete at 
the end of one day or one trial, the same 
as everyone else. 



Q. How was I summoned? 

A. The basic source for names of eligible 
Jurors is the Driver’s License fist which Is 
supplemented by the voter registration 
fist Names are chosen from these com- 
bined lists by a computer in a completely 
random manner. 

Once in the Courthouse, jurors are 
selected tor a trial by this same computer 
and random selection process. 



ensure that all of the jurors who are 
selected to hear the case wilt be unbi- 
ased, objective and attentive. 

In most cases, prospective jurors will be 
asked to raise their hands when a particu- 
lar question applies to them. Examples of 
questions often asked are: Do you know 
the Plaintiff, Defendant or the attorneys m 
this case? Have you been involved in a 
case slmflar to this one yourself? Where 
the answer is yes, the jurors raising hands 
may be asked additional questions, as 
the purpose is to guarantee a fair trial for 
all parties. When an attorney believes 
that there Is a legal reason to excuse a 
juror, he or she wfll challenge the juror for 
cause. Unless both attorneys agree that 
the juror should be excused, the Judge 
must either sustain or override the chal- 
lenge. 

After all challenges for cause have been 
ruled upon, the attorneys wfll select the 
trial jury from those who remain by exer- 
cising peremptory challenges. Unlike 
challenges for cause, no reason need be 
given for excusing a juror by peremptory 
challenge. Attorneys usually exercise 
these challenges by taking turns striking 
names from a list until both are satisfied 
with the jurors at the top of the list or until 
they use up the number of challenges 
allowed. Challenged jurors and any extra 
jurors wfll then be excused and asked to 
return to the jury selection room. 

Jurors should not feel rejected or insulted 
If they are excused for cause by the Court 
or peremptorily challenged by one of the 
attorneys. The voir dire process and 
challenging of jurors Is simply our judicial 
system’s way of guaranteeing both par- 
ties to a lawsuit a fair trial. 



Q. How is the Jury for a particular trial Q. Am I guaranteed to serve on a jury? 
selected? 

A Not all jurors who are summoned actually 
A When a group of prospective jurors is hear a case. Sometimes all the Judges 

selected, more than the number needed are still working on trials from the previ- 

for a trial are called. Once this group has ous day, and no new jurors are chosen, 

been seated in the courtroom, either the Normally, however, some new cases begin 

Judge or the attorneys ask questions. every day. Sometimes jurors are chal- 

This Is called voir dire. The purpose of lenged and not selected, 

questions asked during voir dire is to 
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DOCUMENT LITERACY, LEVEL 1 (0 TO 225) 



Almost one in four 
adults are at Level 1 in 
Document Literacy. 
They can match infor- 
mation and fill in 
personal information 
on forms. 



AM — t a 




They can: 

locate a piece of information based on a literal 
match between the task and the document or 
enter information from personal knowledge onto 
a document. Little, if any, distracting information 
is present.* 

They are likely to succeed at tasks such as: 

• Sign their names (69) 

• Locate time of meeting on a form ( 180) 

• Use a pie graph to locate type of vehicle having 
specific sales (214) 



However, some were at the lower end of this level where they could 
not perform even these routine tasks. 



EXAMPLE TASK 



You have gone to an employment center for help in finding a 
job. You know that this center handles many different kinds of 
jobs. Also, several of your friends who have applied here have 
found jobs that appeal to you. 

The agent has taken your name and address and given you 
the rest of the form to fill out. Complete the form so the 
employment center can help you get a job. 

Birth date Age Sex: Male Female 

Height Weight Health: 

Last grade completed in school 

Kind of work wanted: 

Part-time Summer 

Full-time Year-round 





: Jy 
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DOCUMENT LITERACY, LEVEL 2 (226 TO 275) 



Almost three in 10 
adults can match 
information in docu- 
ments that contain 
dis tractors, and they 
can integrate informa- 
tion from several 
places in a document. 




They can : 

match a single piece of information, with distracting 
information present, or requiring a low level of 
inference. They may also integrate information from 
various parts of the document. 

They are likely to succeed at tasks such as: 

• Locate an intersection on a street map (230) 

• Locate eligibility from table of employee 
benefits (246) 

• Identify and enter background information 
on application for social security card (259) 



EXAMPLE TASK 

What is the gross pay for this year to date? 
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DOCUMENT LITERACY, LEVEL 3 (276 TO 325) 



Three in 10 adults are 
at Level 3 in Document 
Literacy. They can 
integrate several 
pieces of information 
and deal with rather 
complex tables and 
graphs that contain 
information not per- 
taining to the task. 



AS — — — #J1 



•# M 




They cam 

integrate several pieces of information from one or 
several documents and deal with rather complex 
tables or graphs which contain information that is 
irrelevant or inappropriate to the task. 

They are likely to succeed at tasks such as: 

• Identify information from bar graph 
depicting source of energy and year (277) 

• Use sign out sheet to respond to call 
about resident (298) 

• Enter information given into an auto- 
mobile maintenance record form (323) 



EXAMPLE TASK 



You need to smooth wood in preparation for sealing 
and plan to buy garnet sandpaper. What type of 
sandpaper should you buy? 




ExtraCoarM C » Coarse M . Medium F . Fine VF 

SAFETY INFORMATION: ■ list particle /dust mask or other 

I Wear approved safety goggles to prevent inhalation of 



when sending. 



sanding dust. 



• Very Pine EF ■ Extra Fine SF ■ Super Fine UF ■ Ultra Fine 



■ When using power tools, follow 
manufacturer's recommended 
procedures and safety instructions. 
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DOCUMENT LITERACY, LEVEL 4 (326 TO 375) 



Almost one in seven 
adults are at Level 4. 
They can do tasks 
requiring greater 
inferences and 
involving more 
detailed information. 



They can: 

perform tasks that require them to draw higher level 
inferences and numerous responses without being 
told how many are needed. They can also perform 
tasks that contain conditional information. 

They are likely to succeed at tasks such as: 

• Identify the correct percentage meeting specified 
conditions from a table of such information ( 342 ) 

• Use a bus schedule to determine appropriate bus 
for given set of conditions ( 352 ) (See below) 

• Use a table of information to determine 
pattern in oil exports across years ( 352 ) 



EXAMPLE TASK 

On Saturday afternoon, if you miss the 2:35 bus leaving Hancock and Buena Ventura going to 
Flintridge and Academy, bow long will you have to wait for the next bus? 

A Until 2:57 p.m. B Until 3:05 p.m. C Until 3:35 p.m. D Until 3:57 p.m. E I don’t know 




ROUTE 

5 



VISTA GRANDE 

This but Rne operate* Monday through Saturday providing local service" 
to most neighborhoods In the northeast section. 

Buses run thirty minute* apart during the morning and afternoon iush hours Monday through Friday. 
Buses run one hour apart at el other times of day and Saturday. 

No Sunday, hofldsy or night service. 



OUTBOUND 



from Terminal 
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INBOUND 
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2:45 


230 


3*3 


3:15 


3:15 


327 


3:42 


3:47 


3:57 


i££ 


6*6 


6:15 


320 


3:33 


3:45 


3:45 


3:57 


4:12 


4:17 


427 


320 


335 


3:45 


3:50 


4:03 


4:15 


4:15 


427 


4:42 


4:47 


437 


1 330 


4*5 


4:16 


420 


4:33 


4:45 


4:45 


4:57 


6:12 


4:17 


527 


420 


435 


4:45 


430 


5*3 


5:15 


5:15 


5:27 


5:42 


5:47 


537 


4:50 


5*5 


5:15 


520 


633 


6:45 


6:45 


5:57 


6:12 


6:17 


627 


520 


535 


5:45 


530 


6*3 


6:15 












6:50 


6:06 


6:16 


620 


633 


6:45 












620 


635 


6:45 


630 


7*3 


7:15 














Sir 



Y ou cm tra n sfer fr om IN» but 
Id mother /MsdW anywhere 
•tot *? th§ dty bus system 



Arrive 

Downtown 

Terminal 



7:15 

7:45 

6:15 

6:45 Monday Ovoegh Mday a*y 
9:15 
0:45 
10:15 

10:45 Monday S w o* » Wtd fey s Q 
11:15 
12:15 
1:15 pm 



2:15 

3:15 

4:15 

4:45 M o n day M w y > frrtdoy aady 
5:15 

6:45 Monday WWW so 
6:15 
6:45 



***** 
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DOCUMENT LITERACY, LEVEL 5 (376 TO 500) 



One in 33 adults are 
at Document Level 5. 
They can use complex 
documents that 
contain distracting 
information and make 
high level inferences. 



They can; 

search through complex displays that contain several 
pieces of distracting information, make high level 
inferences from the text, and make use of specialized 
knowledge. 

They are likely to succeed at tasks such as: 

• Use information in a table to complete a graph 
including labeling axes ( 378 ) 

• Use a table to compare credit cards; 

Identify the two categories used and write two 
differences between them ( 387 ) 

Use a table depicting information about parental 
involvement in a school survey to write a para- 
graph summarizing the extent to which parents 
and teachers agree ( 395 ) (See below) 

example task 

Using the information in the table, write a brief paragraph summarizing 

stltJ^^sZh^l paretUS and teac bers agreed or disagreed on the 
statements about issuespertaining to parental involvement at their school 




Parents and Teachers Evaluate Parental 
Involvement at Their School 



Do you agree or disagree that . . .? 



Level of School 



Total Elementary Junior High High School 



Our school does a good job of 
encouraging parental involvement in 
sports, arts, and other nonsubject areas 



percent agreeing 









Our school does a good job of 
encouraging parental involvement in 
educational areas 













Our school only contacts parents 
when there is a problem with their child 





Our school does not give parents the 
opportunity for any meaningful roles 




Source: The Metropoftan life Survey ot the American Teacher. 1987 
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QUANTITATIVE LITERACY, LEVEL 1 (0 TO 225) 



Over one in five adults 
are at Quantitative 
Literacy Level 1. They 
can perform single, 
relatively simple 
operations. 




They can: 

perform a single, relatively simple arithmetic opera- 
tion, such as addition. The numbers to be used are 
provided, and the operation to be performed is 
specified.* 

They are likely to succeed at tasks such as: 

• Total a bank deposit entry (191) (See below) 

• Level 1 is from 0 to 225; some are at the lower end of this level where 
they cannot perform even these routine tasks. 



EXAMPLE TASK 



You wish to use the automatic teller machine at your 
bank to make a deposit. Figure the total amount of 
die two checks being deposited. Enter the amount 
on the form in the space next to TOTAL. 

Availability of Deposits 

Funds from deposits may not be available for Immediate withdrawal. Please refer to 
your institution's rules governing funds availability for details. 

_ — — — — — — — — — — — — — — — — , 
Crediting of deposits and payments is subject to verification and collection of actual amounts 
deposited or paid In accordance with the rules and regulations of your financial institution. \ 



PLEASE PRINT 



I 



YOUR MAC CARD NUMBER (No PINs PLEASE) 

lit 222 3331 

YOUR FINANCIAL INSTITUTION 

tynioH&cuik 



YOUR ACCOUNT NUMBER 

987 555 674 



YOUR NAME 



CHECK ONE 



□ DEPOSIT 

or 

□ PAYMENT 



CASH 


$ 


00 


LIST CHECKS 
BY BANK NO. 


ENDORSE WITH NAME 
A ACCOUNT NUMBER 




557 


19 




75 


00 




















TOTAL 







tD 

YC 

to 

Si 

o 



i 



DO NOT FOLD NO COINS OR PAPER CUPS PLEASE 



O - 

ERIC 



21 



23 BEST COPY AVAILABLE 



QUANTITATIVE LITERACY, LEVEL 2 (226 TO 275) 



One in four adults are 
at Level 2. They can 
perform an arithmetic 
operation using num- 
bers given to them or 
easily located in the 
material. 




They cam 

perform a single arithmetic operation using numbers 
that are given in the task or easily located in the 
material. The operation to be performed is either 
described in the task or easily determined from the 
format of the material (for example, an order form). 

They are likely to succeed at tasks such as: 

• Calculate postage and fees for certified mail (238) 

• Determine the difference in price between tickets 
for two shows (.246) (See below) 

• Calculate the total costs of purchase from an order 
form (270) 



EXAMPLE TASK 



The price of one ticket and bus for "Sleuth” costs 
how much less than the price of one ticket and bus 
for “On the Town”? 

THEATER TRIP 

A charter bus will leave from the bus stop (near the Conference Center) 
at 4 p.m., giving you plenty of time for dinner in New York. Return trip 
will start from West 45th Street directly following the plays. Both theaters 
are on West 45th Street. Allow about 1 Vi hours for the return trip. 

Time: 4 p.m., Saturday, November 20 

Price: “On the Town” Ticket and bus $11.00 

“Sleuth” Ticket and bus $8.50 

Limit: T\vo tickets per person 
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QUANTITATIVE LITERACY, LEVEL 3 (276 TO 325) 



Three in 10 of all 
adults are at Level 3. 
They can solve prob- 
lems with two or more 
numbers that must 
be found in printed 
material. 




They can : 

perform tasks where two 
or more numbers are 
typically needed to solve 
the problem, and these 
must be found in the 
material. The operation(s) 
needed can be determined 
from the arithmetic 
relation terms used in the 
question or directive. 

They are likely to 
succeed at tasks 
such as: 

• Use calculator to 
calculate differences 
between regular and 
sale price from an 
advertisement (278) 

• Use calculator to 
determine the discount 
from an oil bill if paid 
within 10 days (308) 

• Calculate miles per 
gallon using informa- 
tion given on mileage 
record chart (321) 



0 




EXAMPLE TASK 

Suppose that you took the 12:45 p.nu bus from UjLLR. Student Union to 
17tb and Main on a Saturday. According to tbe schedule, how many 
minutes is tbe bus ride? 



South Highland 








BUS LEAVES 
from 
UJLLR. 
Student Union 


Bus ant vet 

at 

20th & 
Woodrow 


Bus arrives 

at 

17th & 
Main 


BUS ENDS 
at 

Capitol & 
Louisiana 


WEEKDAYS | 


Ail 


X 


538 


551 


630 


639 






831 


825 


835 


645 




A 


0:41 


655 


735 


7H5 




V-A 


7.31 


725 


735 


745 




& 


7:41 


755 


835 


5*15 






8:11 


825 


835 


845 




X 


8:41 


656 


935 


5115 




V-A 


9:14 


927 


930 


945 




X 


9*44 


9:57 


1038 


T0T15 




o 


1034 


1027 


1038 


1045 




X 


HEW 


1037 


1136 


TTTT5 




Ca 


11:14 


1127 


1136 


11:45 




JL 


11:44 


1157 


1238 


12S15 


PM. 


Cx 


1934 


1227 


1239 


1246 




X 


1Z44 


1£87 


139 


1=13 






134 


127 


139 


1X9 




X 


144 


137 


23V 


£19 




La 


234 


227 


239 


249 




X 


2X4 


237 


£08 


£18 




1a 


334 


327 


239 


2X6 




X 


943 


33V 


438 


4113 




1a 


433 


429 


439 


4X9 




X 


443 


431 


535 


5=15 




lA 


S33 


529 


639 


6X6 




X 


5515 


53V 


837 


1=17 




LA * 


831 


922 


930 


_ 




& 


8X8 


927 


735 


- 


SATURDAY | 


AM 


_£. 


538 


531 


630 


839 




& 


8:45 


637 


739 


7:15 




_£_ 


7:45 


737 


838 


6:15 




£ 


845 


837 


936 


9rl5 






9:45 


937 


1039 


1035 




£ 


1045 


1037 


1138 


11:15 




£ 


11:45 


1137 


1238 


1236 


PM 


£ 


1248 


1237 


139 


135 




_£_ 


146 


137 


239 


239 




£ 


248 


237 


239 


239 




£ 


348 


337 


439 


436 




£ 


446 


437 


939 


£16 




£ 


<46 


637 


939 


838 




£ 


844 


939 


739 


- 
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QUANTITATIVE LITERACY, LEVEL 4 (326 TO 375) 



One in six of all adults 
are at Level 4 in Quan- 
titative Literacy and 
can handle two or 
more arithmetic 
operations in 
sequence. 




They can: 

perform two or more operations in sequence or a 
single operation in which the quantities are found in 
different types of displays, or where the operations 
must be inferred from the information given or from 
prior knowledge. 

They are likely to succeed at tasks such as: 

• Determine correct change using information in a 
menu (33 V 

• Use information stated in news article to calculate 
amount of money that should go to raising a child 
( 350 ) 

• Use the eligibility pamphlet to calculate the yearly 
amount a couple would receive for basic supple- ' 
mental security income (368) 



EXAMPLE TASK 

Estimate the cost per ounce of the creamy peanut 
butter. Write your estimate ou the line provided. 




Unit price 




v You pay 


1.59 per ib. 




^ 1.99 


creamy pnt butter 


10732 °| 


IH 


20 02. 


ir jin£ow 







2 € ' ‘ 




QUANTITATIVE LITERACY, LEVEL 5 (376 TO 500) 



One in 25 of all adults 
perform at Quantita- 
tive Level 5. They can 
perform multiple 
arithmetic operations 
sequentially. 




They can: 

perform multiple opera- 
tions sequentially. They 
can also find the features 
of problems embedded 
in the text or rely on 
background knowledge 
to determine the quan- 
tities or operations 
needed. 

They are likely to 
succeed at tasks 
such as: 

• Determine shipping 
and total costs on an 
order form for items 
in a catalog (382) 

• Use information in 
news article to calcu- 
late difference in 
times for completing 
a race (405) 

• Use a calculator to 
determine the total 
cost of carpet to 
'■over a room (421) 




EXAMPLE TASK 



You need to borrow $10,000. Find the ad for Home 
Equity Loans on page 2 in the newspaper provided. 
Explain to the interviewer how you would compute 
the total amount of interest charges you would pay 
under this loan plan. Please tell the interviewer 
when you are. ready to begin. 



FIXED RATE • FIXED TERM 


HOME 1 
EQUITY 1 
LOANS 


14 . 25 % 

Annual Percentage Rate 
Ten Year Term 


SAMPLE MONTHLY 


REPAYMENT SCHEDULE 


Amount Financed 


Monthly Payment 


$10,000 


$156.77 


$25,000 


$391.93 


$40,000 


$627.09 


120 Months 14.25% APR 



25 



27 



BEST COPY AVAILABLE 




LIMITING PHYSICAL OR MENTAL CONDITIONS 
AMONG ADULTS IN THE LEVEL 1 POPULATION 



Among 40 million adults in the lowest prose literacy level (Level 1), one in four have some 
kind of physical or mental condition that may impair their literacy proficiency, or make 
improvement more difficult. 









Percentage 



of People in Prose Literacy Level 1 Vyho:Lg|^:|IJ^^^ 






o 

ERIC 



Were bom in another country 
or territory 
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LITERACY AND CITIZENSHIP 



Literacy and the exercise of citizenship in a democracy are related. Higher levels of literacy 
go hand-in-hand with voting or obta inin g information from newspapers and magazines. 



0 

ERIC 




BEST COPY AVAILABLE 



27 



29 








THE PRISON POPULATION 



The prison population is considerably less literate than the general population. Almost seven 
in 10 prisoners are in Levels 1 and 2 of Prose Literacy, compared with less than half of the 
general population. 




Level 5: 

A Ail Adults 



Incarcerated 



10 



20 

Percentage 



30 



~1 

40 
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LITERACY AND EDUCATION 



Prose proficiency rises with education leveL Three-fourths of adults with between zero and 
eight years of school are in Level 1, as are four in 10 high school dropouts. Over 75 percent 
of two-year college graduates, and 85 percent of four-year college graduates, reach at least 
Level 3. However, just 4 and 10 percent, respectively, reach Level 5. 



; ■ . Edu<»tipn^ 

[A ^ffaerage 

Saw 







Some 

Degree (308) 



.e.r» — j ■ - - — - •- • 



, r ^s^g^Year 

Degree (322) 



: Gradfstudies/ 
Degree (336) 



^ "a :: : -v ' r:W- ; 

mW -«;VlJV5E&?t • A 'V •?* • • ; *r. : 



f- 



10 30: 50 70 



10 30 50 70 10 30 50 70 

Percentage at Prose Level 
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literacy and poverty 



Poverty and literacy are closely related; 43 percent of adults in Prose Level 1 are poor or 
near poor, compared with just 4 percent in Level 5. * > °° 













-•vvg f ,' 
• ; v r 



1 ' • ' v 20 ■•‘V 30 40 sp 

•;■ -.V Percentage Who Are Poor or Near poor '7;?*^^ 

'? : r Prose Level 5 c : vBv; *> ‘1 >. 



Prose 

Document 

Quantitative 



10 20 30 40 

Percentage Who Are Poor or Near Poor 
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LITERACY AND WEEKLY WAGES 



Weekly wages and weeks worked per year rise with literacy level; Adults in Prose Level 5 earn 
2 1/2 times adults in Level 1, and work more than twice as many weeks. 
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LITERACY AND AGE 



T dt 4f a8e ad ' llts have low literacy levels than young adults, older adults are 
young adults. 688 However ’ older adu lts also have less education than middle-age or 






BEST 
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LITERACY AND PARENTS’ EDUCATION 



Parents’ education greatly influences the literacy proficiency levels reached by their children. 
Even among respondents with similar levels of education, those with better educated parents 
have higher prose literacy proficiencies.* 



* Also true for 
Document and 
Quantitative 



3-cy 






LITERACY, OCCUPATIONS, AND RACE/ETHNICITY 



The average prose proficiency of Black, Asian, Hispanic/Puerto Rican, Hispanic/Cuban, 
Hispanic/Central and South American, and Hispanic/Mexican adults is lower than the 
average proficiency of all laborers. 



ERjt 



Average Prose Proficiency 






500 Bace/Ethnicity •• 
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In Conclusion 



While the more than 
forty million Americans 
performing in Level 1 
on all three scales are a 
diverse group, they are 
characterized in the 
NCES report, Adult 
Literacy in America, as 
at best performing 
“simple, routine tasks 
involving brief and 
uncomplicated texts and 
documents.” Others in 
Level 1 were described 
as “unable to perform 
these types of tasks, and 
some had such limited 
skills that they were 
unable to respond to 
much of the survey.” 
Current literacy pro- 
grams are trying to reach 
this group, primarily 
individuals who can 
hardly read at all. Others 
in this group do display 
limited literacy skills. 

Some idea of how low 
these proficiencies are 
can be gained by a 
comparison with the 
average proficiency of 
laborers, where, among 
the occupations, literacy 
demands are the lowest. 
That average of 249 is 
considerably higher than 
even those in the top of 
Level 1. 

When designing 
literacy programs, it is 
important to remember 
that many adults in the 
lowest literacy level are 
able to decode the 
printed word — able to 
read. Their literacy skills 
are so limited, however, 
that they may need help 
to function satisfactorily 
in even the least demand- 
ins of occupations. 




It is also important to 
keep in mind that they 
are likely to have other 
difficulties, or that they 
are dealing with more 
than one life problem. 
More than four in 10 are 
poor or near poor; one 
in four has a physical, 
mental or health condi- 
tion; and one in five has 
a visual difficulty. 

Another 50 million 
adults are in Level 2. 
They are characterized 
in the Adult Literacy 
in America report as 
“generally able to locate 
information in text, to 
make low-level infer- 
ences using printed 
materials, and to inte- 
grate easily identifiable 
information. Further, 
they demonstrated the 
ability to perform quanti- 
tative tasks that involve a 
single operation where 
the numbers are either 
stated or can be easily 
found in text.” 

It cannot be said that 
these individuals are 
unable to read, as some 
reported after the release 
of Adult Literacy in 
America. But, as this 
report stated, they “were 
much less likely to 
respond correctly to the 
more challenging literacy 
tasks in the assessment — 
those requiring higher- 
level reading and prob- 
lem-solving skills. In 
particular, they were apt 
to experience consider- 
able difficulty in per- 
forming tasks that 
required them to inte- 
grate or synthesize 
information from com- 



plex or lengthy texts or 
to perform quantitative 
tasks that involved two 
or more sequential 
operations in which the 
individual had to set up 
the problem.” 

The Level 2 range of 
scores (226 to 275) 
encompasses the aver- 
age proficiencies of 
adults in laborer and 
craft occupations 
(although many in these 
groups exceed the 275 
level). Also, 29 percent 
of sales workers score in 
the Level 2 range. The 
question for them is 
whether they are vulner- 
able to changes in 
industry that would 
require the more 
advanced proficiencies 
characteristic of Level 3, 
or whether they should 
seek training and educa- 
tion that would make 
them less vulnerable in 
an era of downsizing 
and constantly changing 
labor markets. 

There are, however, 
limits to what we can 
conclude about the 
literacy needs of adults. 
The National Adult 
Literacy Survey was the 
most complete and 
comprehensive literacy 
survey made in the 
United States, and per- 
haps anywhere. But it 
was not a study of the 
levels needed to perform 
particular jobs, or to be 
competent consumers, 
or to function in particu- 
lar contexts. That would 
be a very useful com- 
panion effort; we would 
then be able to place 

37 



occupations or other life 
activities on the three 
scales and be able to 
compare people’s 
proficiencies with the 
literacy needs and 
requirements of these 
occupations. 

Although we lack this 
type of information, 
there is much we can 
see clearly from this 
national assessment 
about the consequences 
of lower literacy pro- 
ficiencies. We can see 
that this low level 
proficiency is not uni- 
formly spread through- 
out the population, and 
that some populations 
are much more at risk of 
low wages and employ- 
ment difficulty. They are 
also less likely to partici- 
pate in community, 
society, and the political 
system than others. 

While one in five adults 
are at Prose Level 1, this 
is true for over half of 
most of the Hispanic 
subgroups and almost 
two in five Black adults. 
This contrasts with one 
in seven White adults. 

While one in four 
White adults are at prose 
Level 2, 37 percent of 
Black, 39 percent of 
American Indian, and 
32 percent of Hispanic/ 
Puerto Rican adults are 
at this level. Three out of 
four, or more, of some 
minority populations are 
at Levels 1 and 2. These 
much lower literacy 
levels are a serious 
obstacle to achieving 
economic parity and 
equal participation in the 
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society. While we do 
not know the literacy 
requirements of particu- 
lar activities from this 
large scale assessment, 
we do know that there is 
a very close relationship 
between literacy profi- 
ciency and success in a 
variety of pursuits, as 
can be seen in the pages 
of this report. 

• Over two in five adults 
in Prose Level 1 are 
poor or near poor, as 
are over one in five 
who are in Level 2. 
Only one in 20 in 
Level 5 are poor or 
near poor. 

• The average weekly 
wages of those in 
Prose Level 1 are 
$240, compared to 
$281 in Level 2, $339 
in Level 3, $465 in 
Level 4, and $650 in 
Level 5. 

• Forty-five percent of 
those in Prose Level 1 
have not voted in a 
state or national 
election during the 
previous five years, 
compared to 11 
percent in Level 5. 

These, and other 
demonstrated relation- 
ships in the full report, 
established clearly that 
literacy proficiency 
serves as a currency in 
economic society, a 
comparison made by the 
study director, Irwin S. 
Kirsch. The more literacy 
you have, the greater the 
economic success, and 







the greater the participa- 
tion in the political sys- 
tem. This is a matter on 
which the data from the 
survey are unequivocal. 

While our greatest 
concern is with increas- 
ing this currency in the 
lower level literacy 
groups, the benefits of 
increasing literacy profi- 
ciency are demonstrable 
for the higher groups as 
well. Improving literacy 
is not solely a matter of 
fighting poverty and 
dependence. The defini- 
tion of literacy for this 
assessment was not only 
to “function in society,” 
but also “to achieve one’s 
goals,” and “to develop 
one’s knowledge and 
potential.” That requires 
continued opportunity to 
increase one’s literacy, 
and to advance in occu- 
pation, income, and 
other aspects of life. 

While formal educa- 
tion is related to 
increased literacy as 
shown in this and other 
surveys, literacy skills 
play a powerful role in 
improving educational 
opportunities. Those who 
read a lot in school are 
better readers; adults 
regularly engaged with 
the printed word are 
more literate. And literacy 
increases for years after 
formal schooling is 
completed, pointing to 
how use begets profi- 
ciency. Those who 
continue to read and 
grow will not lament as 
did Samuel Johnson: “In 
my early years I read 
very hard. It is a sad 



reflection, but a true one, 
that I knew almost as 
much at eighteen as I do 
now.” 

The findings of this 
literacy survey are not 
just for policymakers and 
program designers then; 
they should speak to 
individuals, both about 
the power of being 
highly literate, and the 
personal practices associ- 
ated with getting there. 
One in five adults who 
scored in Prose Level 1 
say they never read the 
newspaper — true 
of only 1 percent in 
Levels 4 and 5. 

While the amount of 
schooling correlates, as 
expected, with literacy 
levels, literacy varies 
widely among people 
with the same number of 
years of schooling. This 
is both because the 
quality of their schooling 
varies and because some 
continue to learn in life, 
while others do not. 
Whichever, the practice 
of expressing adult 
literacy in terms of 
school grade equivalents 
is a prevalent but highly 
misleading practice. 
Having gone to school a 
year at a time, it seems 
that people — those in 
education, in policy, in 
executive positions, in 
creating literacy pro- 
grams, and using literacy 
tests — can think only in 
terms of grade equiva- 
lents. Let’s look at the 
prose skills of those who 
graduated from high 
school, but went no 
further: 



16% are in Level 1 
36% are in Level 2 
37% are in Level 3 
10% are in Level 4 
1% are in Level 5 
When people speak of 
a 12th grade literacy 
level, what can that 
possibly convey? True, it 
takes a shift in thinking 
to embrace measured 
literacy proficiency, as 
done in this assessment, 
and to use levels and 
scale scores to describe 
performance, but not to 
do so leads to wrong 
answers and misguided 
policy. As we said at the 
outset, to work in this 
field one must take some 
time to become literate 
about literacy. 

The difference 
between school-based 
measures of reading and 
the profiles of adult 
literacy used in this 
survey goes beyond even 
the two factors cited 
above. Typical instruc- 
tion in school reading is 
centered on decoding 
words and simply com- 
prehending what is read, 
usually literature. Literacy 
in adult life means 
considerably more than 
that. The printed word is 
constantly serving up 
problems to solve in 
negotiating all aspects of 
life. We can’t use the 
results of this literacy 
survey to measure the 
quality of schooling until 
schools are teaching, 
more directly, what such 
literacy surveys are 
measuring. 

This difference 
between school reading 
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achievement and life 
literacy skills can be 
illustrated using data 
collected in 1985 in the 
Young Adult Literacy 
Survey, also carried out 
by Educational Testing 
Service, with a grant 
from the National Center 
for Education Statistics. 
That survey used the 
same definition of 
literacy, and the same 
three scales, as the 
current one. In addition, 
however, young adults 
were also given the 
in-school reading assess- 
ment used by the 
National Assessment of 
Educational Progress 
(NAEP) for 11th graders. 
The results can be 
compared. NAEP, at that 
time, defined “adept” 
readers as those with . 
scale scores around 300 
(on a 0 to 500 scale). 
Adept readers are good 
school readers who 

can understand 
complicated literary 
and informational 
passages . . . they cart 
also analyze and 
integrate less familiar 
materials and provide 
reactions to and 
explanations of the 
text as a whole. 

A band of readers 
around the adept level, 
those scoring from 275 
to 325 on the NAEP 
scale, could also be 
assessed on the docu- 
ment scale. While 48 
percent of these good 
NAEP readers scored 



0 




from 275 to 325, the rest 
were widely dispersed. 
Twenty-three percent 
scored below the 275 
level, where only rela- 
tively simple tasks can 
be performed, such as 
locating information on a 
pay stub, entering data 
on a deposit slip, and 
matching items on a 
shopping list to coupons. 
Only about 1 2 percent 
scored up at the top of 
the document literacy 
scale (350 or higher), 
succeeding at complex 
literacy tasks. 

It is no wonder that 
schools often turn out 
graduates they think 
meet reading standards, 
while employers (or 
prospective employers) 
of some of these same 
students find them 
grossly deficient at the 
literacy tasks demanded 
by the workplace. The 
schools and the employ- 
ers are not talking about 
the same kinds of tasks 
or competencies. 

The question we need 
to address is where the 
responsibility should lie 
for instilling this broader 
view of literacy, and 
how communication 
between schools and 
employers can be im- 
proved. Further, the 
training provided to new 
employees in America is 
about the most meager 
of the countries with 
which we compete. That 
is also part of the literacy 
problem in America. 

••• 



This first comprehen- 
sive assessment of the 
literacy skills of adults in 
the U.S. is a giant step in 
creating the knowledge 
base for moving toward 
the goal set by the 
President and the gover- 
nors in 1989, that all 
Americans will be literate 
by the year 2000. Beyond 
that we need to spend 
some time making our 
own judgments about 
what this mass of data 
means ... as individual 
citizens, as educational 
institutions, as employers, 
and as public officials. 
Now that we know the 
literacy proficiency of 
individuals, we need to 
know more about such 
things as the literacy 
requirements of jobs and 
what is needed to be a 
competent consumer. 

This too can be gleaned 
from applying existing 
techniques such as job 
literacy analysis. 

Literacy does not come 
easily. And a comprehen- 
sive survey of our state of 
literacy does not translate 
to a couple of sound 
bites. This report at- 
tempts to boil down this 
mass of data as much as 
possible. A more com- 
plete story is in Adult 
Literacy in America: 

A First Look at the Results 
of the National Adult 
Literacy Survey. 
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Print Use Profiles 



By Judith Shaul Norback 
and Garlie A. Forehand 



I. General Introduction 

The purpose of this paper is to examine the literacy demands of jobs. 
Recent research has yielded increasing understanding of the levels and 
processes of adult literacy. The National Adult Literacy Survey (Kirsch & 
Jungeblut, 1986, Kirsch, Jungeblut & Campbell, 1991) analyzes skills of prose, 
document, and quantitative literacy (PDQ), based on a cognitive and linguistic 
analysis of tasks used to assess levels of adult literacy (Ibid., 1991). PDQ 
characterizes literacy tasks in terms of the increasing levels of processing 
required to perform progressively more complex tasks with a given set of 
materials. Examples of PDQ levels and exemplars of those levels are given in 
Table 1. This paper uses the PDQ framework to focus on the literacy demands of 
jobs. 

Norback and her colleagues (1990, 1994, 1995) have collected data on the 
literacy requirements of real jobs. The data includes literacy materials and tasks 
deemed important by job incumbents and supervisors. An example of material 
used in a specific job, in this case, food service worker, is presented in Figure 1. 
The tasks that a worker performs using that material include: 
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reading and following directions on production sheets 
tracking, calculating or recording number of portions 
prepared, served, or left over. 



This paper links actual job literacy requirements with prose levels', 
document levels, and quantitative levels. The resulting literacy profile is 
referred to as a print-use profile. In a print-use profile, job literacy tasks and 
materials are assigned to levels. The five-level classification system is modeled 
on the five levels of PDQ. Unlike PDQ, print-use profiles are not based on 
statistical performance data. Instead, the decision guidelines shown in Table 2 
are used to assign the job literacy tasks and materials to gradations within each 
of the five levels. 

This paper will look at print-use profiles for six jobs: 

1. Entry-level secretary [Entry-level secretaries provide 
administrative support to an individual or group of people. They 
word process, edit and proofread letters, tables and graphs. They 
maintain files, distribute materials, and answer phones.] 

2. Food service worker. [Food service workers prepare and serve 
hot and cold food. They also set up and clean up food displays.] 
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Figure 1: Job Literacy Analyis Food Production Order 
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Table 1: Exemplars of PDQ Levels? 
Difficulty Values of Selected Tasks Along the Prose 
and Quantitative Literacy Scale 


Document, 




Prose ; 


Document > 


Quantitative 7 y. ^ 


Level 5 


Interpret a brief phrase c 
from a lengthy news % . 
article. . t 

. V. ’ ‘ TV/ •• * : 


Using a table depicting 
information about parental 
involvement in school ?M.. 
survey to write a paragraph 
summarizing extent to 
which parents and teachers 
■agree.- ; . : 


Using information in news ; 
article, calculate difference : ? 
in times for completing a " 

<race.4;v^';...'^ 


Level 4 :> 

\ s • . ;{.:}£ 
,N ' ^4v^?iv- i v / . • V : '• 


Explain difference between 

• : fii»A ftrno c ~ r \ f a tvi «% 1 ATrao i ; . ■ 


Use bus schedule foiEglilS 


Using information stated Sftj 


•’ • ; : » *' : ^ 


iwp types o i empi oy 
benefits. ' ' 


determine appropriate bus 
for given set of conditions!! 


in news article, calculate 
amount of money that | 

should go to rising a child. 


Level 3 M 

>' < ; V>: '* • * • .. . ■ <. '* 


Read a news article and 
identify a sentence that 
provides interpretation of a 
situation.: 


Use sign Out sheetitp'l^l^ 
respond to call about 
resident. 


Using a calculator, 
determine the discount : : 

from an oil bill if paid 
within 10 days. 


Level 2 

' ^ t ’ Vs •••' ’ ' " 

“ •••'•• »*• 


Locate two features of 
information in sports 
article. 


Locate eligibility from 
table of employee benefits. 


Determine difference in 
price between tickets for fi*j 
two shows. 


Level l i 

Criiirno* T 


Identify country Msfiort^' 
article. : :: : \ 5. 




Total a bank deposit entry. 



Literacy Survey- 1992. 






. Kolstad, 1993. 



Table 2: 

Print - Use Profile Decision Guidelines 





Prose 


Document 


Quantitative 


Level 5 


summarize multiple sources 


troubleshoot using a 
schematic or intersected list 




. • ■& 


generate answer for 
combined list 


put into or change into 
standard format or know 
when something is out of 
line 


; £ ‘£<g j/’ \ I 






* . * *•> * . ^ ’• •/ 


track quantities from 
multiple sources 


Level 4 


generate/create documents of 
/prose ’! 


create intersecting lists ; . “ 

v. ' *; v s' ■ 4 ■ .*{ 


use background knowledges^ 
to interpret quantitative , : 

results in different types of 


'' •/;■ WM^.- r: 


edit intersecting list :/gv> 


infer using a schematic, an Sg 

/jnfprcprtinty lict nr a 


conversion 




. * A < : > ? > » ’ • .. *: . : *: ' /- 4< %V : ’*• ‘ | ' 


liUCloCvllilg llpl) U1 a IlColCUi •..# 

list, or create a combined list? 


. £■ : & r H ’ ✓>' v ' . i s -IK* •' ; 

: . ’ ' • -.f 


-V . :• 


proofreading nested list or 
intersecting list ; : 4?l5fe. . 


infer using a form or ; ■ 

combined list •' 


: multiple operations with 
more than two numbers 


Level? 


proofread simple or combine 
list . . •■ 


generate (write or enter) Hr 
information ■$■?. - f-'.v - 


plot data on a graph 


\ .IVpfg- : ..;; 


look up information in 
standard reference 


read and/or match a nested 
list or schematic \ T: ?)■ . s 


multiple two-number . " ? 

operations 


"•. v .1§£ ; •;.; 


locate information and use it 
to generate response 


read an intersecting list or a 
diagram : 


• ■*.' ' v 

!l ‘ / 5$ ■?/ i 

• ' V. ;‘;c ; ^ 

• - • .-;;r • • ■ i \'. >V-- ; 




generate description of what 
was seen or heard or told 


read and/or match a simple 4 
or combined list •: ./ 


two or mofe sources of 
infbrmatiori £%f. 


Level? 


copy/transfer information 


read a combined 


». >• V. 7 V 


••.••• • ' s •■ 


read prose of technical nature 
(e.g., MSDS sheet) 


generate a simple list or .4 
cycle through a form for 
multiple pieces:of 
information^?;' 


compare numbers on-V f ^^p 
intersecting list 

• . <v ' ■. " .. <.. 4 

; :v : ).!,■:; v/; : , 


■ /_ . ’. ..;:> : ,•:_.• •> ; 


read job-specific vocabulary 
(e.g., teriyaki monkey dish) 


read a simple list 


■ use units; x per y 


% *■** 


read prose in particular order 


copy or transfer information ; 


'.'SfSd^iu - 


Level 1 




■ . . y /~ . 


onecomputatipn 


- > 


read sentences with 
abbreviations and symbols 


<:3-E'K'JX ■■-V 


write or keypunch a numl«r ; > 
or read a percent, fraction, or ; 
a data in numbers - - - '• 




read more complex words 
such as “deluxe,” 
“Hanover” 


* .*• * N > 4- V. '’ '’I 


read a number or time 

W’ \ • .'*• v J;.: ’• > * 4 v£. < 




read common words such as 
“in,” “and” 




count a’ number 

■ -;-m- ' ■ ■ mm# 
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3. Mid- to executive-level secretary 1 . [Mid- to executive-level 
secretaries provide administrative support to an individual or 
group of people. They create systems for easy retrieval on the 
computer or word processor, and word process, edit and 
proofread letters, tables and graphs. They prepare presentation 
materials, arrange meetings and schedule appointments. Higher- 
level secretaries also answer phones, maintain files and distribute 
materials.] 

4. Nurse assistant. [Nurse assistants measure patients' vital signs, 
height, weight, food intake and output. They record the 
information on charts and graphs. Nurse assistants also review 
patients' schedules with their patients and record their patients' 
daily activities.] 

5. Quality control. [Quality control workers help to ensure product 
quality and work environment cleanliness. They run laboratory 
tests and procedures and observe practices used by other 
employees. They keep detailed records of their results and 
observations.] 

6. Sanitation. [Sanitation workers clean and sanitize the production 
equipment. They mix the chemicals they use and disassemble and 
assemble equipment that they clean.] 



II. Use Profiles 



The prose-document-quantitative (PDQ) approach to adult literacy is 
based on a cognitive and linguistic analysis of tasks used to assess levels of 
adult literacy (Kirsch & Jungeblut, 1986, Kirsch, Jungeblut, & Campbell, 1991, 
Kirsch, Jungeblut, Jenkins & Kolstad, 1993). PDQ characterizes literacy tasks in 



1 The criteria used to distinguish between entry-level secretaries and mid to executive-level 
secretaries included current job title, job title of next higher and next lower positions, and 
amount of experience (Norback, Wilson & Wattay, 1990). 




-v\V^0. 



43 



45 



terms of the increasing levels of processing required to perform progressively 
more complex tasks with a given set of materials. The process used to analyze 
actual workplace materials and tasks using the PDQ framework is described in 
Appendix 1. 



III. The Findings 



For each job grouping that was analyzed, the range of the instances, or 
materials used for specific tasks, will be described. 2 A concrete example from 
the actual job materials and tasks will be given for the lowest and the highest 
ends of the range. Then the continuity of the data will be noted— that is, 
whether the PDQ print-use data tended to group into the levels with no breaks, 
or whether breaks occurred as the job materials and activities were matched to 
the PDQ levels. Finally, for each job grouping, the mode or level with the 
highest frequency will be described, along with the number of instances that 
the mode represents. 



2 The number instances analyzed for each job grouping is: entry-level secretary, 286 (147 
instances of prose, 72 document, 67 quantitative), food service worker, 52 (13 prose, 22 
document, 17 quantitative), mid-to-executive level secretary, 419 (142 prose, 141 document, 136 
quantitative), nurse assistant. 111 (40 prose, 40 document, 31 quantitative), quality control, 78 
(26 prose, 26 document, 26 quantitative), and sanitation, 26 (8 prose, 10 document, 8 
quantitative). 
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Entry-Level Secretary 

Prose. The prose demands for the job grouping of entry-level secretary 
range from the higher region of level 1 to the lower region of level 5. An 
example of the low end of the range is reviewing diaries, appointment books, 
calendars and tickler files. An example of the high end of the range is 
compiling summary reports based on notes taken over a period of time or data 
from one or more sources (e.g., a monthly report). As seen in Figure 2, the data 
axe not continuous; there are two breaks. One outlier shows up at level 1, and 
two other outliers occur at level 5. The former is described above. The latter 
include compiling summary reports, and processing (opening, prioritizing, 
sorting, and answering) the mail. The most frequently occurring examples of 
entry-level secretary prose print use occur in level 2, for example, maintaining 
the existing files. The mode represents 30 different instances, or material-and- 
task combinations performed on the job. 

Document. The document print use of entry-level secretaries ranges from 
level 2 (see Figure 2), for example, transferring information directly from 
handwritten receipts to standard requisitions and invoices, to level 4, for 
instance, compiling summary reports from two or more monthly reports. The 
data is continuous, and the mode, which represents 15 instances, falls in level 3. 
An example of the most common use of documents is proofreading checklists, 
worksheets, and time records. 
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Figure 2: Entry-Level Secretary Print Use Profile* 
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Quantitative. For entry-level secretaries, the use of print for quantitative 
purposes ranges from level 1 to the lower portion of level 4 (see Figure 2). An 
example of the lower level is distributing company guidelines, policies, and 
procedures to the appropriate departments. The higher level is represented by 
performing calculations on requisitions and invoices. There are three breaks in 
the range for the quantitative aspect. The outliers occur at levels 3 (e.g., 
revising charts and graphs in reports and newsletters and checking the 
calculations on travel vouchers and purchase orders), and 4, (e.g., performing 
calculations), as described above. The quantitative mode for this job grouping, 
reflecting 44 instances, is in the upper range of level 1. An example of the most 
frequent type of activity is processing or filling out form letters or memos. 

Food Service Worker 

Prose. The prose print use of food service workers is spread from level 1 
to relatively high in level 2. An example of the lower level is reading and 
following instructions on a chart or job description. The higher level includes 
activities such as reading the hazardous material sheet. The distribution of the 
food service worker prose data is continuous, as shown in Figure 3. The mode, 
representing four instances, is relatively high in level 1, and includes, for 
example, reading and following the directions on food production orders. 

Document. Food service worker document print use is shown in Figure 3. 
The data ranges from reading a label, at level 1, to recording portions prepared, 
served, or leftover on the food production order, at the top of level 3. There are 
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three breaks in the data, with the most prominent outlier occurring at level 3, 
for instance, reading and following the directions in a recipe to prepare items. 
The most common activity reflects seven task-and-material combinations and 
falls in level 3. 

Quantitative. Quantitative food service worker print use (see Figure 3) 
ranges from reading and following instructions on job descriptions (the middle 
of level 1) to modifying the recipe as needed to accommodate more or fewer 
people (level 4). There are two prominent outliers appearing at levels 3 and 4. 
The first outlier is tracking and calculating for the food production order the 
number of portions prepared, served, or left over. The second outlier is the 
conversion described above. The mode for food service workers' quantitative 
print use, reflecting 11 cases, falls in level 1. This level represents reading a 
label or reading a menu. 

Mid- to Executive-Level Secretary 

Prose. As shown in Figure 4, the prose use of mid- to executive-level 
secretaries ranges from relatively high in level 1 to level 5. The two ends of the 
range are indicated by, respectively, reading requisitions and invoices to check 
calculations, and compiling summary reports based on logs taken over a period 
of time. The data has two breaks. Outliers include drafting correspondence 
(level 4) and compiling summary reports (level 5). The mode, representing 25 
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Figure 3: Food Service Worker Print Use Profile 
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Level 1 Level 2 Level 3 Level 4 Levels 



instances,, occurs in the middle of level 2. This level is represented by creating 
and maintaining logs and records. 

Document. Document print use of the higher-level secretaries ranges 
from level 2 to the lower portion of level 5, or, for instance, from making 
reservations with travel agencies and hotels, to verifying that company policies 
are followed with respect to requisitions. The data are not continuous; there is 
one break. Outliers such as verifying the following of company procedures 
occur in the low end of level 5. The most frequently-occurring activity falls in 
the higher portion of level 3 and includes editing the content of marketing, 
public relations, and presentation materials. The mode represents 38 instances, 
or material-and-task combinations performed on the job. 

Quantitative. The numerical print use of mid- to executive-level 
secretaries reflects work from level 1, for example, reading diaries and 
appointment books and extracting the pertinent information, to the top of level 
4, for instance, tracking expenses on expense reports. There are four breaks in 
the data, with examples of outliers including filling out a travel form or petty 
cash voucher (level 3) and tracking expenses (level 4), as mentioned previously. 
The mode is in the higher portion of level 1 and represents 72 instances. An 
example of this level is putting meeting minutes and agenda into standard 
format. The mode is in the higher portion of level 1 and represents 72 instances. 
An example of this level is putting meeting minutes and agenda into standard 
format. 
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Nurse Assistant 



Prose. As Figure 5 indicates, nurse assistants use prose that ranges from 
level 1 to the bottom of level 3. An example of the lower level is reading the 
patient's appointment schedule. The higher level is represented by reading a 
fire safety questionnaire. The nurse assistant data has two breaks in it, with the 
prominent outlier located at level 3. The mode, reflecting 15 cases, appears at 
level 2. The level is represented by monitoring and reviewing the patient's 
vital signs record. 

Document. Nurse assistant document use ranges from obtaining the 
patient's or a relative's signature on the Patient's Bill of Rights (level 2) to 
recording the patient's activities on his or her turning record (the top of level 
3). There is one break in the data but no prominent outliers. The mode, 
representing 20 instances, occurs at level 3. 

Quantitative. The quantitative print use of nurse assistants is shown in 
Figure 5. The work activity ranges from level 1, for instance, reviewing the 
daily assignment sheets, to level 3, for example, calculating the shift and daily 
totals for each patient on the intake and output record sheets. The data has two 
breaks, with the outliers at level 3 (e.g., monitoring and recording the patient's 
vital signs, height, and weight and recording intake and output). The most 
frequent quantitative instance appears at level 1, reflecting 28 cases. An 
example of level 1 includes filling out vacation and leave request forms. 
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Figure 5: Nurse Assistant Print Use Profile 
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Quality Control 



Prose. As indicated in Figure 6, the prose demands for the job grouping 
of quality control range from level 1 to level 3. An example of level 1 is 
reading a gauge; an instance of level 3 is reading and interpreting codes and 
sample data in an analysis chart. The data are not continuous, but have two 
breaks. The main outlier occurs in level one. The mode for quality control 
prose falls in level 2, for example, weighing a sample to be tested. The mode 
represents seven different instances. 

Document. Quality control document use is shown in Figure 6. The data 
ranges from recording the results on a test report (the lower part of level 3) to 
studying the test report results to decide the correct amount of ingredients (the 
lower part of level 5). There are two breaks in the data, with the principle 
outliers in level 5. The mode, representing 11 cases, fall in level 3. 

Quantitative. The quantitative print use of quality control is displayed in 
Figure 6. The work activity ranges from level 1 to level 4. An example of the 
lower end of the range includes weighing the amount of a sample, and an 
example of the higher end is averaging the standard amount of a substance and 
dividing by the factor for that week. There are two breaks in the range, with no 
main outliers, but a noteworthy gap in the data at level 2. The most frequent 
instance occurs in level 1 and is represented by the weighing of various 
samples. The mode reflects seven instances. 
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Figure 6: Quality Control Print Use Profile 




seauB)8iJ| jo eOejueojed 



O 

ERIC 



vH 

CO 



55 



Sanitation 



Prose. Prose print use in the sanitation job grouping is concentrated at 
level 2, ranging from identifying items for evaluation on the preoperational 
checklist to reading the information on the Material Safety Data Sheet (MSDS). 
The data is continuous (see Figure 7). The mode, reflecting six instances, occurs 
in the lower part of level 2. 

Document. Document print use in sanitation varies from using the 
control panel to initiate the pumps and turn on the water and steam to 
troubleshooting problems with the product flow using the flow diagram. The 
range covers the higher portion of level 2 to the lower portion of level 5. The 
data has two breaks, with the outliers in levels 4 and 5. An example of level 5 
was described above. An example of level 4 includes using the process 
instrumentation diagram to identify the flows and connections among the 
tanks, pipes and panels. The two most frequently occurring activities, each 
reflecting three cases, appear at levels 2 and 3. An example of the first is given 
above; a case of the latter includes locating in the chemical publications the 
information on safety, mixing, handling, and usage. 

Quantitative. Quantitative print use in the sanitation job grouping, 
shown in Figure 7, varies from mid-level 1 to mid-level 2. Examples of work 
activities demonstrating this range include reading the circulation diagram to 
locate the milk lines and reading the information from the Material Safety Data 
Sheet (MSDS sheet) on the overview of the chemicals. The data is continuous 
and the mode, reflecting five cases, occurs at level 1, described above. 
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Figure 7: Sanitation Worker Print Use Profile 
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IV. Discussion of the Findings 



Prose, Document, and Quantitative Demands of Jobs 

The analysis of the print-use profiles of six job groupings has added to 
our understanding of the PDQ literacy demands of actual jobs. The profiles for 
each job grouping are shown in Figures 2 through 7. The prose demands of the 
job groupings may span one level, as with sanitation, or up to five levels, as 
with entry-level secretary and mid- to executive-level secretary. The document 
print use of the various job groupings tend to span across at least two levels. 
Quantitative print use is focused on level one, with outliers existing in all job 
groupings except sanitation. 

Outliers manifest themselves in most of the quantitative analyses and 
some of the document and prose analyses. When instruction is being 
developed for particular job groupings, the content could focus on the levels of 
the main body of the data. Supplemental instruction could then teach workers 
the specific job tasks that meet the needs reflected by the outliers. For example, 
entry-level secretary "outlier" or adjunct instruction for prose needs would 
center on level 5: e.g., processing (opening, prioritizing, sorting and 
answering) the mail; and compiling summary reports based on two or more 
sources. The main body of instruction would bring workers to prose level 4. In 
sanitation, adjunct instruction for document use would concentrate on level 5, 
for example, troubleshooting problems with product flow using the flow 
diagram, and level 4, for instance, using the process instrumentation diagram 
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to identify the flows and connections among the tanks, pipes, and panels. The 
bulk of the instruction for sanitation would bring participants to document 
level 3. 

Adjunct instruction for the quantitative skills for food service workers 
would teach them how to track and calculate, for the food production order, the 
number of portions prepared, served, or left over; and conversion. These tasks 
represent the higher part of level 3 and the lower part of level 4. In this job 
grouping, since the main body of quantitative print use is in level 1, workers 
may not need a main body of instruction. 

Adjunct instruction for the quantitative demands of the nurse assistant 
position would center on monitoring and recording the patient's vital signs, 
height and weight, and calculating the shift and daily totals for each patient on 
the intake and output record sheets. These tasks represent the higher portion of 
level 3. Again, in this job grouping, most nurse assistant quantitative print use 
is at level 1; a main body of instruction may not be needed. 

Print-use profiles can provide information useful in developing cost- 
effective instruction for the prose, document and quantitative print use of 
various job groupings. It is important that the main body of instruction and the 
adjunct instruction be high in functional context. As early as 1984, Sticht and 
Mikulecky referred to functional context as the use in instruction of a setting 
meaningful to the persons taking the instruction. Many curriculum developers 
have stressed the incorporation in workplace basic skills training of materials 
actually used on the job (Norback, in press). Recent research (e.g., Mikulecky & 
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Lloyd, 1995) supports the thesis that learners will tend to apply skills learned in 
the classroom to the workplace if at least 20 to 30 percent of the instructional 
materials include workplace examples. So the use of contextual or workplace 
materials and tasks in the main body of instruction and the adjunct instruction 
is highly recommended. 

Print-use profiles can also provide helpful guidelines relating to the 
experience expected for the job. For instance, it may be more effective to expect 
food service worker applicants to have level 1 quantitative skills and to instruct 
them specifically in conversion rather than attempt to bring them to level 4 in 
other quantitative skills. 

In the 1992 National Adult Literacy Survey (NALS), more than 26,000 
adults 16 and older were interviewed. Almost one-fourth (21 to 23 percent) 
demonstrated skills in level 1 of prose, document and quantitative 
proficiencies. Another one-fourth (25 to 28 percent) of the respondents 
demonstrated skills in level 2 on each of the literacy scales. An estimated one- 
third (28 to 36 percent) registered in level 3, and a final one-fifth (18 to 21 
percent) fell into levels 4 and 5 (Kirsch, Jungeblut, Jenkins & Kolstad, 1993). 

Yet our data from the workplace indicates that the print-use levels of four of the 
six job groupings reflect use in levels 4 and 5. Within prose use, two of the six 
jobs range into level 4 or 5, and in document and quantitative use, four of the 
six jobs range into levels 4 or 5. The key difference may be the focus on job 
context in print-use profiles. It is possible that even though the NALS survey 
indicates limited levels of general adult literacy, workers may be more 



proficient in their work contexts when working with materials and tasks 
recognizable to them. It is also possible that the survey numbers reflect that 
workers are more proficient in application mode (that is, applying or using 
these skills) than in assessment mode (that is, in being tested on these skills). 
This is one reason that performance assessment and simulation learning have 
received an added emphasis in recent years (Norback, in press). 

Educational Implications 

Implications for training on the job have been discussed. The results are 
equally applicable to education in schools. 

Reading at work is different from reading at school. For example, 
Mikulecky has described differences in reading in school and work; reading at 
school is likely to focus on longer, less varied sources (Diehl & Mikulecky,1980, 
Mikulecky, 1982, Sticht & Mikulecky, 1984, Mikulecky, 1985, Rush, Moe & 
Storlie, 1986). Philippi has pointed out how reading to learn is different from 
reading to do (Philippi, 1988, 1991). 

Teachers need to be knowledgeable about the world of work. They often 
are missing the constructs that would help them understand how literacy skills 
impact job performance. We need to consider how to provide that knowledge. 
For example, employers could periodically send to schools sets of information 
about skills required. Employers could also provide examples of materials and 
tasks that require various levels of prose, document, and quantitative literacy. 
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Analysis of the literacy demands of jobs can provide principles for 
bringing together the world of school and the world of work. Workplace tasks 
and materials can stimulate the learning of prose, document, and quantitative 
skills in school settings. There is increasing recognition of the need for 
expanded relationships between schools and employers. There is renewed 
interest in apprenticeships, job shadowing, and traditional programs of 
cooperative education, in which schools and employers share instructional 
responsibilities (Barton, 1996). In addition, there are precedents for employers 
providing instructional material to teachers (e.g., the New Standards Project 3 ). 
On the basis of the analysis of both skills and demands, we offer some 
suggestions to make employer-school collaboration effective. 

• The school-employer interaction needs to be regular and 
systematic. 

• There needs to be rewards on both sides. 

• Schools and employers need to engage in collaborative problem- 
solving to identify the pedagogical implications of workplace 
tasks and materials, and to design effective teaching activities 
using those materials. 

• Both teachers and employers require instruction on how to turn 
workplace materials into effective teaching materials. 



3 Personal communication with New Standards teachers who had collected materials from the 
workplace, July 1994. 




62 

70 



References 



Barton, P. (1996). Cooperative education in high school: Promise and neglect. 
Princeton, NJ: Educational Testing Service. 

Diehl, W.A. & Mikulecky, L. (1980, December). The nature of reading at work. 
Journal of Reading, 221-227. 

Kirsch, I.S., & Jungeblut, A., (1986). Literacy: Profiles of America's young adults. 
Princeton, NJ: Educational Testing Service. 

Kirsch, I.S., Jungeblut, A., Jenkins, L. & Kolstad, A. (1993). Adult literacy in 

America: A first look at the national adult literacy survey. Washington, DC: 
U.S. Department of Education, National Center for Education Statistics, 
Office of Educational Research and Improvement. 

Kirsch, I.S., Jungeblut, A., Jenkins, L. & Kolstad, A. (September 1993). Adult 
literacy in America. Washington, DC: National Center for Educational 
Statistics, Office of Educational Research and Improvement, U.S. 
Department of Education. 

Mikulecky, L. (1982). Job literacy: The relationship between school preparation 
and workplace actuality. Reading Research Quarterly, 17:3, 400-419. 

Mikulecky, L. (1985, March). Literacy task analysis: Defining and measuring 

occupational literacy demands. Bloomington, IN: Indiana University. Paper 
presented at the American Educational Research Association 
Convention, Chicago, IL. ED 262 206, CE 042 604. 

Norback, J. Shaul. (1995, August 31). The Norbackjob literacy structure. Princeton, 
NJ: Center for Skills Enhancement, Inc. 

Norback, J. Shaul. (In press). Literacy skills analysis for job training. Philadelphia, 
PA: National Center on Adult Literacy, Univresity of Pennsylvania. 

Norback, J. Shaul, Rosenfeld, M. & Wilson, S.T. (1990). Identifying the literacy 

requirements of jobs and job literacy analysis: A new methodology. Princeton, 
NJ: Educational Testing Service. 

Norback, J. Shaul, Wilson, S.T., & Wattay, D. (1994). Identifying the literacy skills 
needed by entry-level and mid-to-executive level secretaries. Princeton, NJ: 
Center for Skills Enhancement, Inc. 

Philippi, J.W. (1988). Lessons learned about workplace literacy from military job- 
specific reading programs. Public/ Private Ventures, Philadelphia. Paper 



O 

ERIC 



63 



presented at the International Reading Association Convention, Toronto, 
Ontario. 

Philippi, J. W. (1991). Literacy at work. The workbook for program developers. New 
York, NY: Simon & Schuster Workplace Resources. 

Rush, R.T., Moe, A.J., & Storlie, R.L. (1986). Occupational literacy education. 
Newark, DE: International Reading Association. 

Sticht, T.G. & Mikulecky, L. (1984). Job-related basic skills: Cases and conclusions. 
ERIC Document Reproduction Service No. 246 312. 





64 



72 



Appendix I 

Print-Use Profile Procedure 



Note: Each judgment takes into account both the task performed by the 
worker and the particular material used in the task. For example, for 
mid-to executive-level secretaries, proofreading text in form letters and 
memos has a lower prose demand than proofreading text in reports, 
proposals, newsletters, and magazines. 

1) Start with a description of the most important materials read and 
used on the job, and the tasks performed by the workers with the 
materials. 

2) Select the tasks that are most telling of the PDQ level and that are 
not redundant. Use these guidelines: 

a) when possible, combine tasks within a material 

b) when possible, find one task whose PDQ implications 
subsumes several others, for example, "send to other parties" is 
subsumed under "write down and distribute messages." 

c) when the tasks for one material have similar 

question/ directives (e.g., locate, generate— Kirsch, Jungeblut, 
Jenkins & Kolstad, 1993), they can be combined, for example, 
entry level secretary: send messages/ distribute to appropriate 
parties; process telephone messages. 

d) computer tasks, telefaxing, typing and word processing are 
considered not to fall under PDQ. 

3) Read the general description for each PDQ level and decide which 
level the task and material match. E.g., "Prose level 4— These tasks 
require readers to perform multiple-feature matches and to 
integrate or synthesize information from complex or lengthy 
passages. More complex inferences are needed to perform 
successfully. Conditional information is frequently present in 
tasks at this level and must be taken into consideration by the 
reader." (Kirsch, Jungeblut, & Campbell, 1991, p. 11) 
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4) Read the exemplars for the PDQ levels and decide which exemplar 
the workplace task and material match. For example, see Table 1, 
prose, level 4: "Explain differences between two types of 
employee benefits." 

5) Read the Job Literacy notes on PDQ levels in Table 2 and decide 
which portion of which level the workplace task and material 
match. Record notes on the reasons for the decision. 

6) Note which other workplace tasks and materials fall into that job 
literacy area or group and compare them: are they roughly equal 
in difficulty? 

7) If the workplace task and material match more than one area in a 
level (e.g., if the material contains a simple list and a combined 
list), match them to the harder of the two. Assume that the easier 
is subsumed. 

8) If task requirements are unclear due to the limit of information 
given, pick a reasonable estimate. If the information is so limited 
as to be unclear, do not include the task and material (for instance, 
in Sanitation, the task for Chemical Publications of: "identify 
which chemical to use for the job"). 

9) Check the judgments a second time. 

10) Have another analyst independently review the judgments for 
agreement. Discuss any disagreements and add guidelines to this 
list as necessary. 
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Appendix 2: Prose Print Use Profiles 
Prose Print Use Profiles of Six Job Groupings 
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Prose Print Use Profiles (continued) 
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Appendix 3: Document Print Use Profiles 
Document Print Use Profiles of Six Job Groupings 
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Document Print Use Profiles (continued) 
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Appendix 4: Quantitative Print Use Profiles 
Quantitative Print Use Profiles of Six Job Groupings 
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Quantitative Print Use Profiles (continued) 
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Literacy Levels for 90 Occupations 



by Paul E. Barton 

For different occupations, how proficient do you have to be in dealing 
with instructions, information, and problems delivered in print form? Short of 
the use of job analysis techniques— studying each specific occupation— there is 
no way to establish this with certainty. In this volume Judith Norback and 
Garlie Forehand translate job literacy analyses done for six occupations into the 
levels of print use delineated in the National Adult Literacy Study. This 
approach to prose, document, and quantitative requirements of a spe cifi c job is 
surer than any other. 

Another approach, which provides much less certainty, but is more 
readily accomplished, is to assess the proficiencies in dealing with prose, 
document, and quantitative materials of people who are actually working in 
specific occupations. This provides certainty in knowing the level of 
proficiency for people actually employed in specific occupations. However, 
they may be more or less literate in the use of these materials than they need to 
be, from the standpoint of actual job requirements. Also, the broader the 
occupational definition used, the wider the range of both actual literacy 
requirements, and the wider the range of literacy proficiencies of people 
working in those occupations. 

The 1992 National Adult Literacy Study (NALS), carried out by 
Educational Testing Service under contract with the National Center for 
Education Statistics, measured the literacy of all adults. It also obtained 
information on labor force status and occupation. So we know the literacy 
proficiencies of employed people, by their occupation. 
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NALS indicates the literacy proficiencies of people employed in a 
particular occupation, to the extent that the size of the sample permits. We 
obtained data (unpublished) for about 90 specific occupations, in which there 
were 45 or more people.* What we show in the pages that follow are the 
average proficiencies for each of these 90 occupations. But because these are 
samples, we only know the band of scale scores that contains the true average, 
if all people in that occupation had been assessed. The real average, then, is 
within this band. 



These bands are displayed in Figure 1 for prose proficiency. For 
example, there were 166 laborers in the sample and the average proficiency is 
somewhere between 230 and 262. The band represents the measured average 
(plus or minus two standard errors). The averages and the standard errors can 
be found in table 1.0. The standard errors, and therefore the width of the bands, 
vary with the size of the sample and the degree of variation in scores for each 
occupation. Figure 1 shows this band and the level of literacy in which it is 
located. 

We will not here describe what these levels mean. They are described in 
Becoming Literate About Literacy, reproduced in this workbook. 

The bands for prose proficiency are all provided in Figure 1. From 
tables 2 and 3, the bands for document and quantitative proficiency can be 
quickly calculated, using the standard error: double the standard error, add this 
number to the average, and subtract it from the average. The chances are 95 out 
of a 100 that the true average will be within this band. 



* There were a few more than 90. They were not used either because of high standard errors or 
because the titles were not descriptive enough. 
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The assessment used for 1992 study is available to use with individuals. 
Information about its availability can be obtained by writing or calling. 
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Figure 1: Average (Mean) Prose* 
Proficiency Scores 
(Employed Adults, 1992) 



Proficency Level 



Operators, Fabricators and 
Laborers 

Laborers, except construction (889) 

Hand Packers and Packagers (888) 

Freight, Stock and Materials 
Handlers (883) 

Stock Handlers & Baggers (877) 

Industrial Truck and Tractor Equipment 
Operators (856) 

Bus Drivers (808) 

Truck Drivers (804) 

Production Inspectors, Checkers and 
Examiners (796) 

Assemblers (785) 

Welders and Cutters (783) 

Miscellaneous Machine Operators (777) 

Textile Sewing Machine Operators (744) 

Precision Production, Craft, and 
Repair Operations 

Supervisors, Precision Metal Working (628) 
Electricians (575) 

Carpenters (567) 

Construction Supervisors, N.E.C. (558) 

Automobile Mechanics (505) 

Supervisors, Mechanics, and 
Repairers (503) 



*The chances are 95 out of 1 00 that the 
is within the band indicated on the 
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Farming, Forestry and Fishing 

Sroundskeepers and Gardeners, 
except Farm (486) 
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Service (418) 

Private Household Cleaners and 
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Figure 1 (cont.) 
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Technical, Sales, and Administrative 
Support 

Administrative Support, N.E.C.(389) 

Teachers Aides (387) 

Data- Entry Keyers (385) 

Bank Tellers (383) 

General Office Clerks (379) 

Investigators and Adjusters except 
insurance) (376) 

Insurance Adjusters, examiners 
investigaors (375) 

Expediters (373) 

Stock and Inventory Clerks (365) 

Traffic, Shipping, and Receiving 
Clerks (364) 

Mail Carriers, Postal Service (355) 

Postal Clerks, excluding mail carriers (354) 

Payroll and Time Keeping Clerks (338) 

Bookkeepers, Accounting, and Auditing 
Clerks (337) 

Records Clerks (336) 

Receptionists (319) 

Stenographers (315) 

Secretaries (313) 

Computer Operators (308) 

General Office Supervisors (303) 

Sales Support Occupations, N.E.C. (285) 

Street and Door to Door Sales 
Persons (277) 
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Figure 1 (cont.) 



Technical, Sales, and 
Administrative Support, (cont.) 

Cashiers (276) 

Sales Counter Clerks (275) 

Sales Workers, Apparel (264) 

Sales Representatives, Mining, 
Manufacturing Wholesale (259) 

Real Estate Sales (253) 

Sales Supervisors and Proprietors (243) 

Legal Assistants (234) 

Electrical and Electronic 
Technicians (213) 

Helath Technologists and Technicians, 
N.E.C. (208) 

Licensed Practical Nurses (207) 

Clinical Laboratory Technologists and 
Technicians (203) 

Managerial and Professional 

Designers (185) 

Lawyers (178) 

Social Workers (174) 

Counselors, Educational and 
Vocational (1 63) 

Teachers, N.E.C. (159) 

Teachers, Special Education (158) 

Teachers, Secondary School (157) 

Teachers, Elementary School (1 56) 
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Figure 1 (cont.) 



Managerial and Professional 
(cont.) 

Teachers, Pre-Kindergarten and 
Kindergarten (155) 

Registered Nurses (96) 

Physicians (84) 

Computer Systems Analysts and 
Scientists (64) 

Electrical and Electronic Engineers (55) 

Personnel, Training, and Labor Relations 
Specialist (27) 

Accountants and Auditors (23) 

Managers and Administrators (22) 

Managers, Properties and Real Estate (18) 

Managers, Food Service and Lodging (17) 

Administrators, Education and Related 
Fields (14) 

Managers, Marketing, Advertising and 
Public Relations (13) 

Financial Managers, (7) 
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TABLE 1.0 



Proficiency Scores by Occupation Codes 
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Occupation code 


783 


68 


448 


252 


( 8.6) 


251 


( 8.7) 


257 


(10.6) 


Occupation code 


779 


47 


284 


244 


(10.1)! 


238 


(10.1)! 


244 


(11-6)1 


Occupation code 


777 


101 


770 


252 


( 8.3) 


247 


( 9.1) 


256 


( 9.3) 


Occupation code 


744 


89 


695 


230 


(11.6) 


223 


( 9.7) 


225 


(9.1) 


Occupation code 


628 


133 


1,006 


279 


( 7.5) 


279 


( 6.4) 


285 


( 7.0) 


Occupation code 


579 


54 


460 


253 


(10.8)! 


247 


(12.5)! 


255 


( 9.7)! 


Occupation code 


575 


56 


370 


283 


(10.4) 


281 


(12.2) 


300 


( 9.4) 


Occupation code 


567 


119 


1,069 


266 


( 7.3) 


262 


( 8.6) 


275 


( 7.4) ? 


Occupation code 


558 


58 


466 


272 


( 8.5) 


275 


( 9.6) 


287 


( 9.0) 


Occupation code 


505 


91 


704 


267 


( 6.6) 


266 


( 7.1) 


273 


( 6.1) 


Occupation code 


503 


52 


464 


293 


( 8.9) 


292 


( 7.2) 


301 


( 8.3) 


Occupation code 


486 


102 


915 


232 


(16.1) 


233 


(12.1) 


234 


(16.6) S 


Occupation code 


479 


88 


755 


227 


(15.4)! 


227 


(16.8)! 


230 


(18.0)! 


Occupation code 


473 


105 


729 


259 


( 8.2)! 


258 


( 9.9)! 


286 


(10.0)! 


Occupation code 


468 


51 


332 


273 


(11.4) 


260 


(12.3) 


257 


(12.2) 


Occupation code 


466 


93 


724 


279 


( 9-7) 


267 


( 9.5) 


271 


( 8.4) 


Occupation code 


458 


108 


587 


265 


( 7.0) 


259 


( 6.8) 


268 


( 7.7) 


Occupation code 


453 


289 


2,139 


236 


( 5.6) 


231 


( 5.7) 


231 


( 6.0) 


Occupation code 


449 


95 


571 


229 


( 8.8) 


229 


( 8.5) 


225 


( 8.3) | 


Occupation code 


447 


306 


1 ,788 


250 


( 4.8) 


246 


( 6.2) 


247 


( 6.1) i! 


Occupation code 


446 


90 


541 


265 


( 7.9) 


261 


( 8.1) 


259 


( 8.1) ? 


Occupation code 


444 


117 


828 


254 


( 6.7) 


250 


( 6.4) 


245 


( 6.5) 



n = sample size; WGT N = population size estimate / 1,000 (the sample sizes for subpopulations may not add up to the total sample sizes, 
due to missing data); PROF = average proficiency estimate; (SE) = standard error of the estimate (the true population value can be said to 
be within 2 standard errors of the sample estimate with 95% certainty). 



! Interpret with caution - the nature of the sample does not allow accurate determination of the variability of this statistic. 
Source: U.S. Department of Education, National Center for Education Statistics, National Adult Literacy Survey, 1992. 
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Proficiency Scores by Occupation Codes 



OCCUPATION CODES 


n 


WGTN 
(/1 .000) 


Occupation code 


443 


47 


419 


Occupation code 


439 


69 


646 


Occupation code 


438 


45 


429 


Occupation code 


436 


240 


1,824 


Occupation code 


435 


221 


1.790 


Occupation code 


434 


47 


345 


Occupation code 


433 


118 


844k 


Occupation code 


426 


90 


725 


Occupation code 


424 


51 


375 


Occupation code 


418 


57 


389 


Occupation code 


407 


118 


688 


Occupation code 


406 


77 


554 


Occupation code 


389 


154 


947 


Occupation code 


387 


93 


565 


Occupation code 


385 


96 


490 


Occupation code 


383 


56 


350 


Occupation code 


379 


164 


938 


Occupation code 


376 


56 


396 


Occupation code 


375 


62 


42(k 


Occupation code 


373 


72 


404 


Occupation code 


365 


87 


717 


Occupation code 


364 


72 


657 


Occupation code 


355 


45 


312 


Occupation code 


354 


50 


328 


Occupation code 


338 


45 


257 


Occupation code 


337 


285 


1,941 


Occupation code 


336 


53 


383 


Occupation code 


319 


117 


903 


Occupation code 


315 


55 


33# 


Occupation code 


313 


431 


2,662^ 



SCALES 


Prose 


Document 


Quantitative 




PROF ( SE) 


PROF ( SE) 


PROF ( SE) 


255 (13.6)! 


254 (13.0)! 


247(11.8)! 


270 (17.4) 


264 (12.0) 


267 (10.2) 


276 ( 6.9) 


277 ( 9.2) 


273 ( 6.5) 


250 ( 6.3) 


249 ( 5.9) 


245 ( 5.4) 1 


285 ( 3.9) 


284 ( 4.6) 


282 ( 5.2) 


297 (10.6)! 


294 (10.6)! 


297 ( 9.1)! 1 


285 ( 5.6) 


283 ( 6.3) 


281 ( 6.2) 1 


272 ( 8.8) 


268 ( 7.7) 


274 ( 6.4) i! 


292 ( 5.6) 


292 ( 6.7) 


296 ( 7.0) S 


304 ( 6.3) 


303 ( 6.9) 


310 ( 7.8) i 


232 ( 7.8) 


222 ( 9.1) 


227 ( 8.5) | 


269 ( 7.9) 


263 (11.3) 


265(11.6) 


306 ( 4.8) 


299 ( 5.4) 


299 ( 5.1) l 


301 ( 8.0) 


293 ( 6.5) 


294 ( 7.6) | 


287 ( 6.6) 


285 ( 5.4) 


289 ( 6.8) p 


304 ( 7.3) 


298 ( 7.5) 


301 ( 7.9) 1 


301 ( 3.5) 


289 ( 6.5) 


292 ( 5.5) = 


307 ( 7.1) 


305 ( 7.5) [ 


311 ( 7.5) 


308 ( 7.0) 


305 ( 7.1) 


312 ( 6.3) 1 


309 ( 6.9) 


304 ( 7.4) 


305 ( 9.6) p 


285 ( 6.4) 


281 ( 6.2) 


285 ( 6.6) 


270 ( 7.5) 


267 ( 8.5) 


276 ( 6.7) 


288 (11.0) 


286 ( 9.9) j 


296 (10.7) 


286 ( 6.0) 


279 ( 6.6) 


286 ( 6.8) 


294 ( 8.5) 


290 (10.8) 


298 ( 9.3) 


303 ( 4.1) 


299 ( 3.9) 


304 ( 4.6) 


290 ( 6.6) 


287 ( 5.2) 


287 ( 6.6) 


305 ( 6.7) 


293 ( 5.6) 


293 ( 5.3) 


284 ( 8.4) 


284 (10.2) 


285(10.8) | 


302 ( 2.6) 


295 ( 3.0) 


299 ( 3.3) | 



JJ“ h P° pulatl0n si2e estimate / 1 ,000 (the sample sizes for subpopulations may not add up to the total sample sizes 

h« 8 J^£ ' PR0 L^ avera9e . proficiency estimate : (SE) = standard error of the estimate (the true population value canbe said to 

be within 2 standard errors of the sample estimate with 95% certainty). 

Interpret with caution - the nature of the sample does not allow accurate determination of the variability of this statistic. 

Source: U.S. Department of Education, National Center for Education Statistics, National Adult Literacy Survey, 1992. 



O 

ERIC 



83 



nna 

a Proficiency Scores by Occupation Codes 



OCCUPATION CODES 








SCALES 


1 






Prose 


Document 


Quantitative 


















WGT N 










n 


(/1 .000) 


PROF ( SE) 


PROF ( SE) 


PROF ( SE) CP. 


Occupation code 308 


68 


449 


287 ( 7.2) 


287 ( 6.5) 


291 ( 8.6) 


Occupation code 307 


45 


280 


285 ( 9.8)! 


281 ( 8.6)! 


292 ( 9.1)! 


Occupation code 303 


146 


956 


304 ( 5.1) 


302 ( 4.3) 


306 ( 5.3) i! 


Occupation code 285 


51 


426 


289 (12.4) 


284 (12.2) 


292 (16.4) 


Occupation code 277 


61 


417 


272 (10.0) 


264 (13.9) 


266 (14.0) ! 


Occupation code 276 


327 


2,610; 


272 ( 3.8) 


271 ( 3.9) 


274 ( 3.2) | 


Occupation code 275 


52 


355: 


284 (10.7) 


285 (10.6) 


291 (12.2) | 


Occupation code 274 


186 


1.476 


285 ( 5.5) 


279 ( 4.4) 


286 ( 5.2) | 


Occupation code 264 


46 


345 


285 ( 7.4) 


284 ( 7.9) 


283 ( 7.7) I 


Occupation code 259 


158 


1 .222 


312 ( 4.4) 


302 ( 4.9) 


316 ( 5.4) | 


Occupation code 257 


47 


290 


319 ( 8.9)! 


299 (11.4)! 


314 ( 9.7)! 1 


Occupation code 254 


88 


761 


320 ( 6.3) 


309 ( 6.6) 


327 ( 7.2) 


Occupation code 253 


68 


528 


313 ( 6.8) 


307 ( 6.3) 


321 ( 5.7) 


Occupation code 243 


379 


3,105 


295 ( 3.9) 


288 ( 4.1) 


298 ( 5.1) 


Occupation code 234 


53 


345 


324 ( 6.1) 


323 (10.3) 


314 ( 9.3) 1 


Occupation code 229 


51 


300 


327 ( 9.9)! 


331 ( 8.0)! 


328 (10.2)! 


Occupation code 213 


62 


382? 


• 306 ( 8.4) 


317 ( 8.2) 


315 ( 8.0) i 


Occupation code 208 


84 


5015 


296 ( 8.6) 


292 ( 9.3) 


287(10.7) 


Occupation code 207 


76 


515 


300 ( 6.9) 


289 ( 7.2) 


292 ( 5.9) f 


Occupation code 203 


66 


420 


311 (10.2) 


303 (11.9) 


304 (14.7) 


Occupation code 185 


58 


436 


292 ( 6.7) 


287 ( 7.4) 


295 ( 6.6) 


Occupation code 178 


88 


641 


353 ( 6.7) 


346 ( 7.7) 


353 ( 5.9) 


Occupation code 174 


115 


668 


326 ( 6.6) 


320 ( 7.0) 


321 ( 6.9) 


Occupation code 163 


47 


294 


309 (12.1) 


299 ( 8.9) 


298 (12.1) 


Occupation code 159 


81 


619 


312 ( 7.3) 


304 ( 7.7) 


312 ( 8.3) 


Occupation code 158 


55 


389 


333 ( 8.7) 


326 ( 7.9) 


333 ( 8.9) 


Occupation code 157 


157 


960 


332 ( 4.5) 


323 ( 5.0) 


327 ( 5.2) 


Occupation code 156 


248 


1.650 


332 ( 3.2) 


321 ( 2.7) 


327 ( 3.3) 3 


Occupation code 155 


46 


2965 


326 ( 7.7) 


317 ( 7.1) 


317 ( 9.8) 


Occupation code 95 


281 


1.648 


326 ( 3.6) 


314 ( 3.7) 


313 ( 3.8) I 



n = sample size; WGT N = population size estimate / 1,000 (the sample sizes for subpopulations may not add up to the total sample sizes, 
due to missing data); PROF = average proficiency estimate; (SE) = standard error of the estimate (the true population value can be said to 
be within 2 standard errors of the sample estimate with 95% certainty). 



I Interpret with caution - the nature of the sample does not allow accurate determination of the variabSity of this statistic. 
Source: U.S. Department of Education, National Center for Education Statistics, National Adult Literacy Survey, 1992. 
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Proficiency Scores by Occupation Codes 



OCCUPATION CODES 










n 


WGTN 
(/1 ,000) 


Occupation code 


84 


48 


367 


Occupation code 


64 


121 


878 


Occupation code 


55 


"il'62. 


- 448 


Occupation code 


27 




SC 426 ; 


Occupation code 


25 


103 j 


|#685: 


Occupation code 


23 


143 


f 1,020 


Occupation code 


22 


'f '464 


>3,715 


Occupation code 


18 


■'.iti. 57 ■ 


;\-3>387: 


Occupation code 


17 


58 


452 


Occupation code 


14 


: '^-67j> 


>•' ;40i 


Occupation code 


13 


89 ' 


641 


Occupation code 


7 


84 


549 



SCALES 


Prose 


Document 


Quantitative 


• \t. "" .• . ". : 




. PROF ( SB 


PROF ( SE) 


PROF ( SE ) ~ >. 



335 (10.4) 
350 ( 5.5) 
348 ( 6.5) 

329 (11.6) 

330 ( 5.0) 
342 ( 4.2) 
316 ( 3.5) 
301 ( 7.2) 
285 (10.1) 
325 ( 8.7) 
338 ( 8.1) 
335 ( 5.4) 



324 (10.1) 
343 ( 5.3) 
343 ( 5.8) 
318 (10.7) 
323 ( 4.7) 
335 ( 5.2) 
311 ( 3.3) 
294 ( 7.7) 
278 ( 9.1) 
J300 ( 9.0) 
330 ( 8.1) 
326 ( 7.4) 



335 (11.4) 
349 ( 6.8) 
367 ( 6.1) 
324 ( 

327 ( 

347 ( 

323 ( 

299 ( 

296 ( 

315 (11.3) 
335 ( 6.2) 
339 ( 6.4) 



9.1) 

5.5) 
6 . 0 ) 

3.5) 

7.1) 
9.3) 



! Interpret with caution - the nature of the sample does not allow accurate determination of the variability of this statistic. 
Source: U.S. Department of Education. National Center for Education Statistics. National Adult Literacy Survey. 1992. 
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IN SEARCH OF COMMONALITIES 



INTRODUCTION 

As a part of the Institute for Educational Leadership’s Center for Workforce 
Development transitional support project, a search to identify "commonalities" among skill 
standards of the 22 pilot projects and three independent skills projects' was undertaken. The 
purposes for such an undertaking are multiple. It is well recognized by most observers that 
one of the key reasons for the growing support of a voluntary national skill standards system 
is a desire to better communicate to all education and training providers, parents, and 
students the core knowledge and skill requirements needed to succeed in the workplace. A 
central expectation of the skill standards "community" is that they will be able to identify the 
common foundation of knowledge and skills needed across the majority of occupations 
within the economy. 

There is not yet a generally accepted precise nomenclature around which one can discuss 
what is meant by the term skill standard. However, there is increasing acceptance that it is 
essential to use a taxonomy that differentiates a hierarchy of knowledge and skills 
requirements. At least three broad categories of skills are recognized as important; these 
include: 1) basic (academic and workplace basics); 2) cross-functional (essentially those 
identified by the SCANS effort and considered to be those which exist in most occupations 
across the economy); and 3) occupation specific skills. Obviously a task centered on the 
issue of commonalities among skill standards would focus on the first two categories. That 
has been the focus of this work. 

To find commonalities, we built upon the work among the 22 pilot projects through the 
findings of a three stage Delphi processes conducted in 1994-95. The purpose of that Delphi 
process was to identify and document where there was agreement and disagreement around a 
common framework. It was recognized at the time the Delphi process was undertaken, some 
framework features would be driven by technical principles and others by philosophy. It was 
anticipated that it would be more difficult to gain consensus on certain features than others. 
Three aspects of skill standards were identified as the focus of the consensus-building 
process: 1) the meaning of a skill standard; 2) the information presented in a skill standard; 
and 3) the organization of skill standard information. Wherever possible the definitions that 
were being used to update and refine the Dictionary of Occupational Titles (now called 
0*NET) were used during the Delphi exercise. 

A summary of the findings of the consensus building process (that included state education 
officials as well as representatives from all of the 22 pilot projects) are found on Table 1. A 
study of that table reflects general support for recognizing different types of skill categories. 
Yet, precisely what is a basic attribute versus a cross-functional versus an occupation 
specific remained unsettled among the 22 projects — and others. They also present the 
greatest challenge in minimizing duplication of skill information since many basic and cross- 
functional skills relate to different skill standards within the same occupational area. 



1 Tha three other projects are: The Academy of Residential Construction Carpentry Skill Standards, the 
Telecommunications Skill Standards project, and the 8anking Skill Standards project. These three are included because they 
are industry sponsored projects which focus on career-entry and basic skills. 
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Table 1 



Term: Definition 



Work Segment: 

In%i£'¥' s 'n r li °/ work to which the standard 
— applies. It defines what has to h* Hnn* 

Essential Knowledge/Skills: 

'norfnrm^rP an< * abilities that are critical to 
penormanc e of specific work 

Workplace Basic Skills 

^J^/nenfa/ developed abilities 
0 f-3), jading, mathematical 
application) that are required by 
specific work and that occur in 
some degree in virtually all jobs. 



B. 



work 



Support Rating 



3.5 



3.8 



3.6 



3.3 



Percent Preferring 
Definition 



42% 



48% 



32 % 



35 % 



Cross-Functional Skills: 

specific wnrk Inn* to performance of 
i wofft and that tend to occur across 

r c% a Ji^ w,de ranges of jobs S roccup^onal 

Occupation -Specific Knowledge: 

A ‘ awareness, or familiarity 

molkrPit facts, principles, processes, 
methods, or techniques tor specific work. 

B. The principles needed to work with or 
v£ori<. 9 9 lven in performing specific 

Occupation-Specific Skills: 

n% t rf%™£!i v * ies or Procedures that are 
performed for specific work 

Performance Criteria. 

The measures and standards used to judge 
the ability of the worker to perform specific 
work. 



3.7 



3.5 



3.4 



3.5 



3.6 



58% 



39 % 



39 % 



48 % 



47 % 
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Methodology for this Study 

As of March 1996 the source documents from which the statements were drawn were 
considered the final core standards documents from each project with the exception of the 
Grocer’s and Laborers/AGC projects. To date the work that has been undertaken has 
organized 10 tables of material that has focused on the category of basic skills, using the 
current working definition of 0*NET, see Table 2. 0*NET has developed two types of 
basic skills categories -- content and process. The content areas, (Mathematics, Science, 
Writing, Speaking, and Active Listening) , sometimes fit the definitions used by the skill 
standards projects of basic skills as they are closely linked to academic knowledge and skill 
fields. The process skills (Critical Thinking, Active Learning, Learning Strategies, and 
Monitoring) are often not a basic skill classification within the skill standards projects. 
However, upon review of the occupation specific skill requirements of many of the skill 
standard projects there was often found statements that directly related to these type of 
process skills. Therefore, it became necessary, to undertake the more ambitious task of 
reviewing the occupation-specific skill sets or lists to develop the basic skills category — 
the category of skills needed by entrants into the workforce. 



Basic 



Content 

ACTIVE LISTENING 

MATHEMATICS 

READING 

COMPREHENSION 

SCIENCE 

SPEAKING 

WRITING 



Process 

ACTIVE LEARNING 

CRITICAL THINKING 

MONITORING 

LEARNING 

STRATEGIES 



Table 2 

0*NETs SKILLS FRAMEWORK 

Cross Functional Occupation-Specific 

PROBLEM SOLVING (VARIED CATEGORIES] 

SOCIAL 

TECHNICAL 

SYSTEMS. 

RESOURCE MANAGEMENT 



The materials that are presented include material culled from both basic and occupation- 
specific statements published by the national skill standards projects (and three other 
state/industry projects) under the basic skills categories planned for 0*NET. Cross- 
functional skills have not yet been culled from the materials. 
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While many of the included statements are considered by the individual standards 
projects to be "standards", the term "statement" is used because some projects have not 
distinguished clearly between their standards and related advisory statements about skills, 
abilities, attributes etc. Thus it should not be assumed that all of the materials contained 
herein are project standards, merely that they are contained within standards documents 
from each project. 

There are a host of difficulties -- bumps in the road, if you will -- that a mapping 
exercise of this sort confronts. Among these are project differences in standards language, 
numbers of standards levels, and degree of specificity in language within each level. The 
limitations of the model being mapped against also figures in the bumpiness. In attempting 
to find standards statements culled from various categories assumptions had to be made 
about what project statements say and what they imply, especially in those instances in 
which the language is vague or is very much the jargon of industry. 

The process for placing the statements was comprised of the following. For the basic 
skills category the standards documents of the- projects were reviewed for categories 
labelled basic skills, foundation skills, employability skills or the like. If statements under 
these headings were found that also linked to 0*NET basic skills categories (Active 
Listening, Mathematics, etc.) they were, in general, placed in the basic skills column under 
the appropriate 0*NET category. Because a basic skill within 0*NET is defined to be 
"capacities" that "promote or provide a foundation for learning other types of material" this 
was even done even in some cases where the skill was worded in an occupation-specific 
way. Thus while the popular sense of basic skills might be the Three R’s at, perhaps, the 
eighth grade level, basic skills to 0*NET simply means the skills preliminary to learning 
the skills distinctive to an occupation. Thus some skills that are basic for one occupation 
might be occupation or industry-specific for others. This, of course, means that the basic 
skills listed by different projects reflect very different levels of accomplishment within these 
skill areas. 

For occupation-specific skills, project materials were reviewed for statements that 
expressed reasonably purely the essence of the basic skill as applied to the job. Statements 
that clearly implied the skill were occasionally found and used. Statements that only 
suggested the skill were not included. 

Statements were kept whole: a statement containing two concepts that would link to 
different categories was not broken apart. If a statement wrapped reading and mathematics 
together it was not, in general, placed in a category. Exceptions were made so that there 
would be information for some projects that, were the rule imposed, would have none. And 
for the process skills multiple statement grouping were often used because it is difficult to 
illustrate some of these process skills with only one statement (these groupings are indented 
from the column margin). 
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The varied standards structures has made deciding which categories of information to 
include a new challenge for each pilot project document. A few projects made the task 
easy in the basic skills area by having a basic or employability skills area and only 
including material in that category that seems truly basic. Some projects tried to do this but 
ended up including occupation-specific or industry specific material. 



Implications 

WHAT IS A BASIC SKILL? 

All projects addressed the issue of basic skills. Yet, it is clear from the review of the 
material that pilot projects could have benefitted from a common set of definitions 
regarding what is to be called a basic skill. The 0*NET system did not exist at the start of 
the pilot projects’ grants and thus there should be no expectation the categories used by 
0*NET would naturally "fall-out", nevertheless, it will be useful for the projects to examine 
the tables and consider whether some skill areas suggested by 0*NET should be captured. 
Some of the 0*NET basic skills are not found in some of the project documents at all. 
Examples of the learning-related skills, active learning and learning strategies were 
especially rare. Establishing the connection between project statements and the categories 
was easier for the content skills. 

The NSSB as well as the pilot projects and those responsible for the development of 
0*NET should carefully consider the definitions to be ultimately incorporated in the basic 
skills category — what are basic skills? Should both process and content be included? If so 
how can differentiations be made between the knowledge and skills required for entry into 
the occupation versus those same types of skills required for higher level or more 
specialized work. 

IDENTIFYING THE BASIC SKILLS 

A key issue that must be addressed by NSSB and others is how and who should be 
responsible for validating basic skills and knowledge that cross cut a substantial portion of 
jobs within the economy? What will be the most cost effective process to document these 
types of skills as well as the category of cross-functional skills? 

TERMINOLOGY MATTERS 

Agreement will need to be reached regarding the forms of expression. Even for the 
more common content skills basic skills have been expressed in some cases that are with 
such generality that comparisons across project lines are difficult. Wide variation in word 
choice makes establishing connections among different projects basic skills difficult on the 
basis of individual skill statements alone. The contextual details needed to establish the 
connection are usually not provided. It is not clear what a statement such as "Demonstrate 
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good reading skills’’ adds. And there are many statements of such generality in the pilots’ 
documents. 

PRESENTATION MATTERS 

Another contributor to confusion is the lack of clarity about what the standard is — 
or where it is - in a pilot’s publication. While some projects clearly mark a section of their 
document as the standard, more often it is implied that the document contents are, taken 
together, the standard. Also, long lists of tasks that do not provide the reader with a 
context of how the task fit into the various categories of skills even within a single 
occupational area will not provide the necessary guidance educators, trainers and 
perspective workers need. The Delphi Process recommendation that the Skill Standard Unit 
include linked Occupational, Basic and Cross-Functional Skills seems especially wise in 
light of our analysis. 

MORE WORK IS AHEAD 

This effort to find the commonalities of basic and cross-functional skills is but a 
beginning. It is not yet completed. For example, one of the tibvious outcomes of such an 
exercise is to seek and find the common knowledge and skill requirements across the 
various projects. This work has yet to be completed, it will require the attention of a wide 
range of organizations and stakeholders. This document simply is a tool to be used. 
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Introduction 



The Norback Job Literacy Structure is an empirically-based list of skills 
built from the workplace. It was developed as a result of 10 years of research 
which built first upon the Prose, Document and Quantitative categories of the 
national adult assessments done by Educational Testing Service, and then 
expanded as more job groupings were analyzed. To date 39 job groupings have 
been analyzed across 43 settings, including industry, government and health 
care. 



The skills listed can be used along with the tasks and materials from the 
workplace (for example, charts, gauges, graphs, and forms) to improve the 
credibility of customized instruction and to build instruction with high 
functional context (that is, instruction that incorporates the context in which the 
skills will be used). The skills listed in the Job Literacy Structure can also be 
used as a guideline or starting point for a literacy skills analysis. Ideally, a 
literacy skills analysis should include the following six steps: 

1. Review job descriptions to identify possible types of materials used on the 
job 

2. Conduct personal interviews of workers to gather materials and related 
information 

3. Edit and condense the information collected 

4. Identify the literacy skills needed to perform the job tasks using the job 
materials 

5. Have Advisory Committees of workers and supervisors review the results 
to check for completeness 

6. Identify the common skills that are required across job groupings. 



If all of the above six steps cannot be included, the following guidelines are 
important: 

* Has the analyst talked with workers? 

* Has the analyst talked with supervisors? 

* Has the analyst asked both workers and supervisors about future 
requirements? 

* Has the analyst identified types of materials used on the job? 

* Has the analyst identified the literacy skills needed to process the materials 
for the specific job task? 

* Has vocabulary from the materials been incorporated into the curriculum? 



JOB LITERACY CATEGORIES OF SKILLS 



IDENTIFIED TO DATE 



Quantitative 

1 . Formulate Problems 

2. Add and/or Subtract 

3. Multiply and/or Divide 

4. Other Arithmetic Processes 

5. Numbers and Counting 

6. Telling Time 

7. Linear, Weight, Volume and Other Measures 

8. Scales and other Gauge Measures 

9. Geometry 

Document 

10. Select 

1 1 . Process Forms 

12. Process Illustrations 

13. Process Tables 

14. Process Graphs, Pie Charts, Bar Charts 

Prose 

15. Reading 

16. Reference Systems 

17. Vocabulary 

18. Writing, Grammar, Editing, Spelling 

19. Following Directions 

20. Identification 



21. Computer-Related Skills 

22. Synthesizing across Formats 

23. Contingent Decision-Making/Analvsis/Troubleshootine 

24. Basic Communication: Working with Other Parties in the Communication System 

25. Basic Communication: Adjusting to the Limitations of Materials 

26. Basic Communication: Communicating about Actions and Procedures 
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QUANTITATIVE 



Formulate Problems 

1. Infer from printed directions whether task involves adding or subtracting 

2. Infer from printed directions whether task involves multiplying or dividing 
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Add and/or Subtract 



Add (single operation) 

1. Add integers 

2. Add decimals 

3. Add fractions 

4. Add mixed numbers (integers and fractions) 

5. Add printed time 

6. Add linear, weight & volume measures 
Subtract (single operation) 

7. Subtract integers 

8. Subtract decimals 

9. Subtract fractions 

10. Subtract mixed numbers 

1 1 . Subtract printed time 

12. Subtract linear, weight and volume measures 
Estimate 

13. Determine an estimate of a sum or difference (integers, fractions or decimals) 
Combination 

14. Use addition and/or subtraction to solve problems involving two or more of the 
following: integers, mixed numbers, fractions and/or decimals 
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Multiply and/or Divide 



Multiply and/or Divide 

1. Multiply and/or divide integers 

2. Multiply and/or divide decimals 

3. Multiply and/or divide fractions 

4. Multiply and/or divide mixed numbers (integers and fractions) 

5. Multiply and/or divide printed time 

6. Multiply and/or divide linear, weight and volume measures (e.g., liquid, dry, 

degree) 

Estimate 

7. Determine an estimate of a product or quotient (integers, fractions or decimals) 

8. Round integers or decimals 
Combination 

9. Use multiplication and/or division to solve problem involving two or more of the 

following: integers, mixed numbers, fractions and/or decimals 

) 
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Other Arithmetic Processes 



1 . Use calculator for multiplication and/or division 

2. Use calculator for addition and/or subtraction 

3. Determine averages 

4. Solve ratio and proportion problems, including reducing fractions to lower terms 

5. Solve problems that combine more than one process (e.g., addition and multiplication, 

multiplication and division) 

6. Solve problems involving units of measurement that combine more than one process 
(e.g., addition and multiplication, multiplication and subtraction) 

7. Solve problems that require conversion 

8. Read and understand information from scales to solve jnath problems 

9. Use mathematical processes to solve word problems 

10. Read and understand U.S. money (e.g., SO cents is half of a dollar) 



f 1 



o 



204 



Numbers and counting 



1. Read or write numbers in sequence 

2. Read or write a number (digits, integers, fractions and decimals) 

3. Read or recognize the meaning of percentages 

4. Read and understand telephone and telefax numbers 

5. Count by intervals of one, two, five or ten 

6. Match numbers with words used as codes or labels 

7. Determine which number, in a group of numbers, is of greatest or least value 

8. Compare and/or match numbers from two different sources 

9. Sort items based on the numbers they are labeled with 
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Telling Time 

1. Tell time using a clock 

2. Read and understand printed time 

3. Read and understand printed military time 

4. Read and understand the relationship of time on a clock to printed time 

5. Read and understand dates and days of the week 

6. Read and understand foreign dates 
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Linear. Weight. Volume and Other Measures 

1 . Read and understand U.S. standard units of length (e.g., inches, feet). 

2. Read and understand metric system units of length (e.g., meters). 

3. Read and understand U.S. standard measures of weight (e.g., ounces, pounds). 

4. Read and understand metric system measures of weight (e.g., grams) 

5. Read and understand U.S. standard measures of volume (e.g., pints, quarts). 

6. Read and understand metric system measures of volume (e.g., liters) 

7. Read and interpret micrometers, calipers 

8. Read and understand measures of temperature in degrees Fahrenheit. 

9. Read and understand measures of temperature in degrees Centigrade. 

10. Read and understand job-specific measures of pressure (e.g., pounds per square inch 

or PSI). 

11. Read and understand job-specific measures of flow (e.g., gallons/min., 

milliliters/min.) or speed (inches/min., rpms or revolutions/min.). 

12. Read and understand job-specific measures (e.g., weight/foot in pounds). 

13. Read and understand exact job-specific measures (e.g., standard cubic ft/min/., 

pounds per sq. inch gauge) due to working with a vacuum. 

14. Read and understand job-specific measures of electricity: resistance (ohms), 

electrical current (amps) and the line "pressure" (volts). 
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Scales and other Gauge Measures 



1. Read and interpret the markings on a scale 

2. Calibrate a scale 

3. Read and interpret readings on test equipment 

4. Read and interpret a digital or computerized scale 

5. Read and interpret the markings on a gauge 

\ 



O 

ERIC 



208 



152 



Geometry 

1. Read and understand measures in terms of angles (e.g., 90 degrees) 
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Document 



Select 

1. Determine which form in a group of forms is of relevance to the 

problem 

2. Determine which graph or pie or bar chart in a group of graphs or charts is of 

relevance to the problem 

3. Determine which illustration in a group of illustrations is of relevance to the problem 

4. Determine which table in a group of tables is of relevance to the problem 

5. be) (S) Highlight important information through language, underlining, or placement of 

materials 

6. be) (S) Leave out information that may be unimportant, already known, or available 

elsewhere 

7. be) (R) Select important or relevant information to attend to 

8. be) (R) Attend to new or changed information 

9. be) (R) Attend to information needed to tailor actions for a particular customer 

10. be) (S) Use conventional formats (for example, a number and the word^'all.'or 

selecting graphical versus text format) for expressing information appropriately 

11. be) (R) Interpret conventional formats for expressing information 
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Forms 



Locate 

1. Locate the space on a form to enter data 

2. Locate information using a completed form 

3. Find information by skimming or scanning an entire form. 

Copy (locate and transfer! 

4. Transfer or copy exact data from other sources onto appropriate section of a 

form (e.g., copy data from intersection of table columns and rows) 

Cycle 

Generate 

5. Enter, create or generate data into the appropriate spaces on a 

form 

6. Write a description of an activity or transaction in appropriate 

section of a form 

Integrate 

7. Compare information using a completed form to other information internal to 

that form 

8. Verify that information in that space is appropriate 

9. Compare information on the form to information in external source 
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Illustrations (Pictures. Diagrams. Schematics. Maps') 



Cycle (or "locate" if only 1 locate) 

1. Locate and use parts of a picture (e.g., labels, numbers) 

2. Locate and interpret specific details of a diagram 

3. Locate and use parts of a schematic 

4. Locate and interpret specific details of a map 

(e.g., state borders, a mountain range, a particular city) 

Generate 

5. Update a schematic or part of a schematic. 

Integrate 

6. Use a key to locate and interpret information in a picture 

7. Use a key to locate and interpret information in a diagram 

8. Use a key to locate and interpret information in a schematic 

9. Use a key to locate and interpret information in a map 

10. Use a scale to interpret information in a diagram. 

11. Use a scale to interpret information in a schematic. 

12. Utilize information from a picture, diagram, schematic or map to compare data 
with actual objects (picture, diagram, schematic or map) 

13. Utilize information from an illustration to choose a course of action. 

Other 

14. Understand or interpret schematics with text in a foreign language. 

1 5. Find information by skimming or scanning an entire illustration. 
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Tables 



Locate 

1. Locate the appropriate space in the table to enter data 

2. Locate and extract information from a one-column list 

3. Locate and extract information from a two-column table 

4. Locate and extract information from an intersection of a row-by-column table 
Copy 

5. Transfer or enter information from another source onto a table 
Generate 

6. Write a description of an item (or enter information) into the 

appropriate section of a table 

Integrate ' 

7. Utilize information from the intersection of a complex table (i.e., a cell) to 

compare data within the table or chart. 

8. Utilize information from the intersection of a complex table (i.e., a cell) to 

compare data with text, document materials or objects outside the table 

9. Utilize information from tables to choose a course of action or solve a math 

problem 

10. Use a key to locate and extract information from a table. 

Other 

11. Find information by skimming or scanning an entire table. 
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Graphs. Pie Charts. Bar Cham 



Locate 

1. Locate and extract information from a line graph 

2. Locate and extract information from a pie chart 

3. Locate and extract information from a bar chart 

Copy 

4. Transfer or enter information from another source onto a graph or pie or bar 

chart 

Generate 

5. Write a description of an item (or enter information) into the 

appropriate section of a graph or pie or bar chart 

6. Plot data on a graph. 

Integrate 

7. Compare information in a graph or chart to other information 

internal to that graph or chart 

8. Compare information on the graph or chart to information in an 

external source 

9. Utilize information from graphs or pie or bar charts to choose a course of action 

or solve a math problem 

10. Use a key to locate and interpret information in a graph or chart. 



O 

ERIC 



158 



Prose 



Reading 



Locate 



1. Match information in the task with information in the material 



2. Underline or circle a portion of the text (e.g., a sentence) 

Cycle 

3. Find information by skimming or scanning text. 

Integrate 

4. Integrate or pull together (e.g., synthesize, summarize) two or 

more pieces of information located at different points in the text 
or reference source 

5. Compare/contrast information in the text with information in an 

external source (a task or other source) 

6. Determine a word's meaning from the context of a sentence or 

paragraph 

7. Determine the main idea of a paragraph or several paragraphs 

8. Read and understand the main idea of a job-specific material 

9. Determine the main idea of a paragraph or several paragraphs which were 

written in non-standard English 

Generate/Inference 

9. Process information in the text but also go beyond that 

information by drawing on own knowledge or by making broad text- 
based inferences 

10. Identify and also use appropriate sections of text and visual materials 
to complete a task 
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Reference Systems 



1. Use alphabetic and alphanumeric filing system (e.g., locating 
files, filing information) 

2. Find information by using a table of contents, index, appendix and 

glossary 

3. Find appropriate section of a reference source (e.g., sentence, 

paragraph, heading, table) to answer a specific question 

4. Find information by cross-referencing sources 

5. Determine the main idea of a reference material (e.g., passages, 
letters, articles) 
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Vocabulary 



1. Read and understand common words 

2. Read and understand job-specific words 

3. Read and understand the directions of east/west/no rth/south. 

4. Read and understand abbreviations, contractions, and acronyms 

5. Read and understand symbols and codes (e.g., colon, percent sign, w/, dollar sign) 

6. Read, understand and use the concept of pH and the related range (0 to 14) and the 

concept of turbidity and the related range (0 to 5000+) 

7. Read and understand standard chemistry abbreviations for elements and compounds 

(e.g., Cu, C02). 




217 

161 



Writing. Grammar. Editing. Spelling 



1. Apply rules of grammar (e.g., verb/subject agreement, use of proper tense) 

2. Recognize words needing capitalization 

3. Apply rules for punctuation (e.g., commas, periods, apostrophes, semicolons) 

4. Make wording changes to improve clarity 

5. Spell common words correctly 

6. Spell job-specific words correctly 

7. Use references to check and correct spelling errors 

8. Generate a written document (e.g., letter) 

9. be) (S) Adapt written communication to an audience's "knowledge and needs 

1 0. be) (S) Identify in writing a "point of contact" for further information 

1 1 . be) (S) Tailor written communication to achieve a particular purpose, in a process 

involving more than one person 



Following Directions 



1 . Follow directions to complete a task that includes reading, identifying, observing 

and/or comparing 

2. Follow very specific step-by-step directions to perform a sequence of tasks 

3. Follow directions to complete a task which includes arithmetic operations and/or 

counting (e.g., shipping/receiving) 

4. Follow directions to complete a task which involves entering information into a CRT 

screen 

5. Follow directions to complete a task using a single illustration or a sequence of 

illustrations (pictures, diagrams, schematics, maps) 
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Identification (recognition) 



1. Identify and label objects 

2. Identify objects by particular physical characteristics (e.g., size, shape, color) 

3. Select appropriate course of action by using taste, touch, sight or hearing 
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Computer-Related Skills 



Locate 

1. Locate and transfer (copy) data from a source (forms, gauges, 

scales) into a CRT screen 

2. Locate information using a computer program 
Integrate 

3. Compare information using a computer program 
Generate 

4. Create or generate information using a computer program 
Select 

5. Determine which keyboard in a group of keyboard^ is of relevance to the 

problem. 

Other 

6. Understand die operation of a computer keyboard and/or mouse. 

7. Read and respond to computer commands 

8. Monitor computer screen for changes in data. 
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Synthesizing across Formats 



1. Synthesize information from more than one source (e.g., written, pictorial, oral) to 
complete a task 
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Contin gent Decision-Making/Analvsis/Troubleshooting 



1* Understand what information means and be able to induce or know, based on the 
information available, what happened or what went wrong. 

2. Understand the sequence and cause/effect relation of the activities being monitored, 

e.g., the material flow, or how the slab comes from the caster. 

3. Determine appropriate course of action in a particular situation (If, then) 

4. Make decisions under time pressure. 

5. be) (S) Identify problem and report problem to appropriate person for further analysis 

6. be) (R) Investigate problem, if appropriate, and report on status 
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B asic Communication: — Working with other parties in the communication system to reach 
understanding 



1. be) (R) Address questions to an appropriate point of contact 

2. be) (R) Interpret information in light of who presented it (for example, assess 

credibility of information) 

3. be) (R) Recognize when identifying the specific author of a communication is 

necessary, and when it is not 

4. be) (R) Identify the relevant person to seek clarification or additional information 

from, when a specific "point of contact" is not named 

Basic Communication: Adjusting to the limitations of materials 

5. be) (S) Provide information that may not be prompted for on a material, when 

considered important 

6. be) (S) Evaluate material as a tool for interaction 

Basic Communicatio n : Communicating about actions and procedures 

7. be) (S) Specify problems or the need for action, but do not specify the associated 

actions or procedures 

8. be) (R) Infer needed actions from written or orally communicated information 
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Skill Levels and 
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Secretary’s Commission on Achieving Necessary Skills (SCANS) 
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How was the domain 

identified? What was rated? 
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Foreword 



At all levels of government, education policymakers are confronting 

Z-A immense problems that cry out urgently for solutions. These men 
-Z A. and women — legislators, governors, mayors, school officials, 
and even the President of the United States — generally agree that our 
schools cannot be left to operate unaltered and that the need for reform is 
widespread and immediate. 

Policymakers know, for example, that the growing demand for early 
education is forcing a crisis in that field and that educators of young chil- 
dren now grapple with demands that are straining their resources and 
compelling them to redefine their mission. They listen as employers loudly 
lament the quality of our high school graduates, while investing millions 
of corporate dollars in programs that teach basic skills and workplace com- 
petencies to their newest workers. And they search diligently for programs 
and practices that can reverse our alarming failure to bolster the achieve- 
ment levels of at-risk students. 

But if the problems are numerous and compelling, there is no shortage 
of proposed solutions. Currently, one of the most favored reform strategies 
calls for implementing accountability measures that would more clearly 
define and assess who is responsible for student success and student fail- 
ure. Thus, while the number of programs, suggestions, proposals, and 
techniques for dealing with such specific issues as literacy or achievement 
levels among at-risk youngsters is mind-boggling, many of these ap- 
proaches now contain one or more strategies for holding schools account- 
able for student learning. 

Given the intensity of the school reform debate and the abundance of 
ideas for remedying the Nation's educational ills, it is not surprising that 
many policymakers often find themselves adrift in a sea of uncollated and 
frequently conflicting information that does little to inform decision- 
making. 

In an effort to alleviate this situation and to inform the education 
debate, the Office of Educational Research and Improvement (OERI) de- 
cided last year to commission a series of papers to address those topics 
that policymakers themselves told us were most pressing. 
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We began by surveying the major policymaking organizations and 
asking them to identify which school-related issues they viewed as 
compelling. There was remarkable agreement in the field, and it did not 
take very long to identify those areas most in need of illumination. We 
learned, for example, that policymakers are concerned about improving lit- 
eracy levels and about graduating young people who are prepared to func- 
tion effectively in the modem workplace. We discovered that they are 
seeking strategies to combat the growing crisis in early childhood edu- 
cation and to raise achievement levels among at-risk students. And we 
found that there is a genuine need to clarify the issues surrounding edu- 
cational accountability, so that intelligent decisions can be made about 
how best to hold schools answerable for their performance. 

Thus advised, we sought the most distinguished scholars we could find 
to address significant aspects of these issues, and we succeeded in assem- 
bling a roster of individuals whose expertise on these subjects is un- 
challengeable. Indeed, I am most grateful to Paul E. Barton, director of the 
Educational Testing Service's (ETS) Policy Information Center, and Irwin 
S. Kirsch, research director for ETS' Division of Cognitive and Assessment 
Research, who combined their considerable knowledge and skill to 
produce this paper on literacy and its relationship to workplace com- 
petencies. 

I am also indebted to: 

• Sharon L. Kagan, associate director of The Bush Center in Child 
Development and Social Policy at Yale University, for her paper on 
Excellence in Early Childhood Education : Defining Characteristics and Next- 
Decade Strategies; 

• Michael W. Kirst, professor of education and business administra- 
tion at Stanford University, for his paper on Accountability: Implications 
for State and Local Policymakers ; and 

• James M. McPartland, co-director of the Center for Research on 
Elementary and Middle Schools, Johns Hopkins University, and 
Robert E. Slavin, director of the Elementary School Program for the 
Center for Research on Elementary and Middle Schools, and co- 
director of the Early and Elementary School Program of the Center 
for Research on Effective Schooling of Disadvantaged Students, 

Johns Hopkins University, for their paper on Increasing Achievement of 
At-Risk Students at Each Grade Level. 



iv 



O 

ERIC 




211 



We asked that all the authors approach the subjects within a common 
framework and bring to bear their distinctive perspectives on these impor- 
tant issues. Specifically, we requested that they do four things: 

• Describe the issue or problem being addressed; 

• Discuss briefly pertinent research on the topic; 

• Describe what States and/or other concerned interest groups are 
doing about the issue; and 

• Analyze the implications of current activity — and inactivity — for 
policymakers at the Federal, State, and/or local levels. 

Then, to ensure that this paper — and the others in this "Policy Perspec- 
tives" series — would, in fact, be valuable to the community of policy- 
makers, we invited all of the scholars to participate in a one-day meeting 
where they could present their draft findings at a public forum and then 
engage in small group discussions that provided a unique opportunity for 
face-to-face peer review sessions. Both authors and reviewers were over- 
whelmingly enthusiastic about this process, and all of the papers were re- 
vised to reflect the feedback offered. 

I want to stress, in conclusion, that it is not the purpose of this series to 
supply easy answers or quick-fix solutions to the complex problems 
confronting American education, today. We did not start out to develop a 
set of blueprints with step-by-step instructions for implementing reform. 
Rather, we are seeking to promote the dissemination of knowledge in a 
format we hope will provide policymakers everywhere with new insights 
and fresh ideas that will inform their decision making and translate into 
strategies that will revitalize the ways in which we run our schools and 
teach our students. 



Christopher T. Cross 
Assistant Secretary 
Office of Educational Research 
and Improvement 
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Preface 



T he issues of sufficient "workplace competencies" and of the need 
to improve literacy and employment readiness are no longer con- 
fined to research studies and academic treatises. These are 
national issues, discussed in conferences and board rooms, at education 
summits, and on the front pages of newspapers. Thus, our purpose in 
writing this paper has been primarily informational: to provide those who 
debate, recommend, and decide, with the most accurate information we 
have available. 

Unfortunately, this important debate has often not been well informed; 
the written word in this area is often based on simplistic rhetoric rather 
than on a broader understanding of the complex nature of literacy in our 
society. As a Nation, we are now far beyond the quest for a simple, single 
number of "illiterates." It is time to move on to the harder tasks of 
describing what school leavers and adults know and can do in the daily 
tasks they confront, and then of better understanding what they need to 
know— for the workplace and for a full life that provides opportunity to 
develop their potential. 

We believe good information is a prelude to constructive action. And we 
offer some views on the implications of the facts we present. For assist- 
ance in developing this paper, we are indebted to the panel assembled by 
the U.S. Department of Education's Office of Educational Research and 
Improvement (OERI) to review this document in draft. They are: Christina 
Dunn, Andrew Kolstad, Mary Harley Kruter, Patricia McNeil, Nevzer 
Stacey, Bernadette Toomey, and Lisa Walker. The manuscript was also re- 
viewed by Norman Freeberg. We thank Joanne Antonoff for her word- 
processing skills. We are also indebted to Kathleen Price at OERI who 
worked with us in the publication of this manuscript, and did so with 
competence and good humor. 

The views here expressed do not necessarily represent the views of Edu- 
cational Testing Service or the Office of Educational Research and 
Improvement. 



Paul E. Barton 
Irwin S. Kirsch 
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O ne of the most publicized problems of this decade is what many 
perceive to be a huge literacy deficiency. One author has pro- 
claimed that we are "a Nation of illiterates/'* Data used in 
the national media have been alarming, and the sources of these numbers 
elusive. The truth, meanwhile, is that few systematic measures have been 
taken of literacy, and none has been repeated. Thus, trends in literacy pro- 
ficiency cannot be accurately estimated. 

While the Nation's concern with literacy appropriately encompasses all 
areas of living, the glare of attention often focuses on the workplace. 

Many employers report that they find it increasingly difficult to identify 
and hire literate workers, and the American Society for Training and 
Development (ASTD) tells us that corporations are spending hundreds of 
millions of dollars on remedial education. Not surprisingly, these employ- 
ers are not happy about the state of affairs: they see increasing school 
expenditures and think that the same thing is being paid for twice. 

Obviously, in order to thrive, the Nation's employers must find workers 
they can train. But in the global economy, the concern extends far beyond 
the individual firm: to many we are "a Nation at risk." More and more, 
Americans are realizing that competitiveness of the economy turns on the 
quality of the country's work force, not just on its machines, technology, 
and management know-how. In fact, thinking in the modem corporation 
has gone far beyond the days of Taylorism when a job was broken into 
discrete steps on a production line and an uneducated worker was trained 
in days — or hours. 

The requirements are likely upward, at least in many important sectors 
of the economy and in particular employers' needs. The present techno- 
logical environment has enlarged some workers' responsibilities. The lines 
between workers and supervisors and managers blur as "work teams" or 
"quality circles" are used to raise creativity and productivity. The team 
members put their heads together and solve problems. The members can 
do each other's jobs. They must be flexible. And they have to deal with 
print, often in more complex forms than before. In short, the requirements 
are rising in some areas that are highly visible to employers and to observ- 



*Kozol, Jonathan. Illiterate America. Garden City: Anchor Press, 1985 . 
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ers of the economy in general, and it is against these growing demands 
and expectations that the adequacy of preparation for the entering work 
force must be viewed. 

But while it is against these same rising expectations that we may be 
slipping, still we see no evidence that we are "a Nation of illiterates/' To 
the contrary, we have made progress — this is a basis for optimism — and 
we can make more. 

For example, a standard used just a century ago to judge whether or not 
Americans were literate was a person's ability to sign his or her name. 
Today, all young adults (ages 21-25) can do that. 

The standard of 50 years ago, the era of World War II, was one of read- 
ing at about a fourth-grade level, a huge jump. Currently, 95 percent of all 
young adults meet this criteria. 

And 25 years ago, the frequently used standard of the War on Poverty 
era was an eighth-grade reading level. Now, 80 percent of all young adults 
can read at that level. 

We believe it is not useful in today's complex and varied economy and 
society to establish a single standard— a point we will elaborate on. But by 
all indices — educational achievement and school retention, for example — 
there has been progress in absolute terms. Meeting our needs is a problem 
we can deal with. But a proper definition of the problem and accurate 
information about the literacy condition are essential ingredients for suc- 
cess. Our purpose is to communicate knowledge and information (to the 
extent they exist) that can be used to inform the processes of policy- 
making and the judgments of policymakers. 

We conclude that there is a need now for increased literacy skills, 
particularly among specific segments of the population. We also conclude 
that available projections of literacy and educational requirements for jobs 
of the future are not dependable enough to be the principal basis for 
action. 
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Profiling Young Adult Literacy 



I n this section we will be referring to the concept of literacy and the 
results of measurements based on it used for the Educational Testing 
Service's (ETS) 1986 report Literacy : Profiles of America's Young Adults , 
a publication of the U.S. Department of Education's National Assessment 
of Educational Progress (NAEP). While much of the rhetoric underlying 
the recent national discussion has focused on the number of "illiterates" in 
the population, we reject as simplistic the notion of a single standard for 
literacy, or a single point on a scale which is used to categorize individuals 
as either literate or illiterate. That report illustrates the point with a quote 
from the 1984 National Society for the Study of Education publication. Be- 
coming Readers in a Complex Society .. 



The often heard charge, "Johnny can't read," is a little like saying 
"Johnny can't cook." Johnny may be able to read the directions for 
constructing a radio kit, but not a Henry James novel, just as 
Johnny may be able to fry an egg but not cook a Peking duck. In 
discussing reading in the schools, we must recognize that reading 
involves as wide a range of different types of texts as there are 
types of food. 

As we leave the concept of reading in the school and enter the work- 
place or society in general, literacy skills become even more diversified. 
There are different kinds of materials to read, and different levels of com- 
petencies required, depending on what life task — or workplace task — is 
applied to these materials. The literacy definition used for the 1986 assess- 
ment was "using printed and written information to function in society, to 
achieve one's goals, and to develop one's knowledge and potential." 



Simulation tasks were constructed for assessing literacy proficiency 
among a nationally representative sample of young adults and reported in 
terms of three important and relatively distinct areas: 



• Reading and interpreting prose , as in newspaper articles, magazines, 
and books. This was designated prose literacy. 

• Identifying and using information located in documents such as forms, 
tables, charts, and indexes. This was designated document literacy. 
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• Applying numerical operations to information contained in printed 
material such as a menu, a checkbook, or an advertisement. This 
was designated quantitative literacy. 

All three types of literacy find their applications in various adult con- 
texts including the workplace, and all three, although differing to varying 
degrees, are only moderately related to the reading comprehension exer- 
cises found in school classrooms. Document literacy represents the kinds 
of tasks frequently associated with the workplace, and found in instruc- 
tion books, repair manuals, and tables which require the worker to move 
from a question or problem to an answer or solution. Indeed, several pub- 
lished studies, including some that one of the authors participated in, sup- 
port this view. 

The 1986 survey itself required contacting some 40,000 households to 
locate and assess approximately 3,600 young adults. Each assessment 
interview lasted about 90 minutes, and it included some 100 questions 
about personal characteristics, educational experiences, exposure to the 
labor market, and reading activities. The results were reported on three 
new literacy proficiency scales — prose, document, and quantitative — where 
proficiency was expressed along scales ranging from 0 to 500 . Levels on 
the scales were illustrated by examples of actual tasks that respondents 
estimated to be proficient at that level were likely to be capable of 
performing. 

The sample of young adults assessed included the full range of edu- 
cational levels from high school dropouts to Ph.D.s, as well as representa- 
tive samples of white, black, and Hispanic young adults. While this 
sample encompassed the full range of literacy proficiencies in the popu- 
lation, the current discussion of deficiencies in the workplace generally 
centers on the knowledge and skills entry-level workers need to perform 
even at a basic level, and on the expectation they will be able to benefit 
from the training employers offer to entry workers coming out of the Na- 
tion s high schools. Therefore, while we will summarize the results for the 
full population, we will then provide more detail for high school grad- 
uates, and also look closely at young adults in various occupations. 

Included in the overall findings were the following: 

While the overwhelming majority of young adults adequately per- 
form tasks at the lower levels on each of the three literacy scales, 
sizable numbers do not do well on moderately complex tasks. 

The earlier that young people terminated their education, the less 
likely they were to attain moderate or high levels of proficiency. 
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Black young adults perform considerably below white young adults; 
Hispanics perform about midway between. 

Only about 2 percent of the young adult population were estimated 
to have such limited literacy skills that the interviewer judged that 
the simulation tasks would unduly frustrate or embarrass them. 
Approximately half of this 2 percent were judged to have limited 
proficiency in English. 

Reading exercises were also given that were used in the regular in- 
school NAEP assessments. Ninety-five percent of young adults 
were estimated to read at or above the average level of fourth-grade 
students. 

Two principal conclusions were: 

That while "illiteracy" is not a major problem in the young adult 
population, the demonstrated levels of "literacy" are a problem. 

There are large percentages who perform only in the middle range 
on the literacy scales. 

That while the overwhelming majority are able to perform routine 
or uncomplicated tasks, relatively small proportions can do mod- 
erately complex tasks. A great many of these young adults will not 
be prepared for the workplaces of the present economy, irrespective 
of what these workplaces may be like in the future. 

For an examination of workplace literacy, it will be helpful to take a 
closer look at high school graduates. While employers may find defi- 
ciencies in young adults with postsecondary degrees, they are much more 
likely to be disturbed about the kind of low-level literacy that they find in 
entry-level workers coming from the high schools. One may assume that 
high school dropouts are, on average, headed for difficulty in the modem 
workplace, and we will also illustrate their literacy levels. 
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Literacy of High School Graduates 



T he prose, document, and quantitative literacy of 21- to 25-year-old 
high school graduates* is summarized in figures 1, 2, and 3. As in 
the case of young adults generally, practically all young adults 
who finished high school are able to use printed information to accom- 
plish tasks that are routine or uncomplicated. 

• For prose literacy, 97 percent performed at least at the 200 level on a 
scale of 0-500. One task characteristic of performance at this level 
is writing a simple description of the type of job one would like to 
have (199) (See figure 1).** 

• For document literacy, 97 percent performed at least at the 200 level 
One characteristic task directs the reader to match money-saving 
coupons to a shopping list of several items (211). Another task in- 
volves entering personal background information on a job applica- 
tion (196) (figure 2). 

• For quantitative literacy, 93 percent performed at least at the 225 level. 

A task that typifies this level requires totaling two entries on a 
bank deposit slip (233) (figure 3). 

While there can be some solace in the finding that almost all high 
school graduates assessed perform at these basic levels, literacy skills seem 
to us distressingly limited: relatively small proportions of young adult high school 
graduates were proficient at levels characterized by the moderate or relatively complex tasks. 

• For prose literacy, just 27 percent performed at or above the 325 level 
A representative task at this level required locating material on the 
basis of three bits of information that were repeated throughout a 
lengthy news article (313). 



•Actually, these numbers Include high school graduates who reported some form of post- 
secondary education but who did not obtain either a 2- or 4 -year degree. These results, 
therefore, somewhat overestimate the average literacy levels of high school graduates only. 

••Tasks used to illustrate these levels are those the respondents had a high probability of 
being able to perform (80 percent); they are chosen to be characteristic of the kinds of tasks 
associated with these levels. 
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Figure 1. — Percentage of high school graduates scoring at or 

above selected levels of proficiency in prose literacy, 
with tasks illustrative of various levels: 1986 



Proficiency 

scale 


Percent at 
or above 
selected 
levels 


Example tasks at various 
proficiency levels 


500 


k 






400- 








375- 




3.2 (0.8) 


371— Orally interpret distinctions between 
types of employee benefits 


350- 




12.2 (1.3) 


340 — State in writing argument made in 
lengthy newspaper column 


325- 




26.6 (1.8) 




300- 




48.4 ( 1 -7) 


313 — Locate information in a lengthy news 
article 


275 - 




66.4 (1.4) 


277 — Write a letter to state that an error has 
been made in billing 








262 — Locate information in sports article 


250- 




81.4 (1.3) 




225- 




91.2 (0.9) 




200- 




96.7 (0.6) 


199 — Write about a job one would like 



NOTE: High School graduates include those who had some postsecondary experience, 
but no degree. Standard errors are shown in parentheses. 



SOURCE: National Assessment of Educational Progress, Literacy: Profiles qf America’s 
Young Adults , 1986. 
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Figure 2. — Percentage of high school graduates scoring at or 
above selected levels of proficiency in document 
literacy, with tasks illustrative of various levels: 1986 



Proficiency 

scale 


Percent at 
or above 
selected 
levels 


Example tasks at various 
proficiency levels 


500 


L 






400- 








375- 




2.6 (0.5) 




350- 




10.9 (1.3) 




325- 




28.0 (1.7) 


334-365 — Use bus schedule to select 
appropriate bus for given 
departures and arrivals 


300- 

275- 




50.2 (2.1) 
70.6 (1.5) 


300 — Follow directions to travel from one 
location to another using a map 
278 — Use index from an almanac 


250- 




83.4 (1.2) 


249 — Locate intersection on street map 


225- 




91.8 (0.8) 


22 1 — Enter date on a deposit slip 


200- 

175- 




96.9 (0.5) 
99.2 (0.2) 


196 — Enter personal information on job 
application 

181 — Enter caller’s number on phone message 
form 


1 -'Ir- 

o o 

iO 




99.8 (0.1) 


160 — Locate expiration date on driver’s 
license 



NOTE. High School graduates include those who had some postsecondary experience, 
but no degree. Standard errors are shown in parentheses. 



SOURCE: National Assessment of Educational Progress, Literacy: Profiles of America's 
Young Adults , 1986. 
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Figure 3. Percentage of high school graduates scoring at or 
above selected levels of proficiency in quantitative 
literacy, with tasks illustrative of various levels: 1986 




NOTE: High School graduates include those who had some postsecondary experience, 
but no degree. Standard errors are shown in parentheses. 

SOURCE: National Assessment of Educational Progress, Literacy: Profiles of America's 
Young Adults , 1986. 
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• For document literacy, only 11 percent were estimated to be at or above 
the 350 level, where they were likely to be able to do tasks such as 
figuring out from a bus schedule the time on a Saturday morning 
when the second bus arrived at the downtown terminal (334). 

• For quantitative literacy, just 30 percent were estimated to be at or 
above the 325 level, where a typical task required the reader to 
examine a menu, to compute the cost of a specified meal, and to 
determine the correct change from a specified amount (337). (Only 
about 13 percent were at the 350 level where performance included 
figuring the exact amount of a 10 percent tip.) 

Needless to say, high school dropouts performed much less well than 
graduates. On the prose scale, just 10 percent were able to find informa- 
tion in the news article, compared with 27 percent for graduates. On the 
quantitative scale, the results were similar. Just 10 percent were able to 
compute the cost of a meal from a menu, compared with 30 percent for 
graduates. 

The above information is gleaned from examining just two levels on 
each of the three scales. Everyone can look at representative tasks at dif- 
ferent proficiency levels and make judgments about what proportions of 
young adults are ill prepared for life's challenges. The levels of literacy 
needed by any individual depend on the demands he or she faces in the 
different life areas of work, home, and community. And within those 
areas, the questions become: What job? Doing what in the home? Doing 
what in the community? The NAEP study has measured what young 
adults can do; it has not measured what different settings require of them. 
For example, among high school graduates, fewer than 1 in 20 is estimated 
to be at or above the proficiency level (375) needed to estimate cost by 
using grocery store unit price labels. Is that acceptable in the present and 
future workplace? 

These levels would seem to be disappointing and inadequate, if society 
requires a more competent labor force in an economy increasingly shaped 
by technology. 

Not only are small proportions of young adults achieving advanced 
levels on the literacy scales, but, as pointed out above, proficiency levels 
vary considerably among different populations of young adults. (The dif- 
ferences described below are based on the entire sample of 21- to 25-year- 
olds, instead of only high school graduates.) 

• Black young adults, on average, perform significantly below white 

young adults on the literacy scales, with Hispanic young adults 



11 




226 



performing midway between. Among young adults, 86 percent of 
blacks, 94 percent of Hispanics, and 98 percent of whites perform at 
east at the 200 level on the prose literacy scale. Eleven percent of 
ac , 24 percent of Hispanic, and 43 percent of white young adults 
perform at or above the 325 level on this scale. 



• The longer the time spent in school, the higher the literacy pro- 
ficiency. Again using the prose scale, registering at or above the 200 
level are 71 percent of those with 8 or fewer years of school 88 per- 
cent of high school dropouts, and 97 percent of the young adults 
who have a high school diploma or some postsecondary education. 
At or above the 325 level are 0 percent of those with 8 or fewer 
years of school, 10 percent of high school dropouts, 27 percent of 
those with a high school diploma or some postsecondary education 
and 63 percent with a postsecondary degree. 



While the use of simple print for routine tasks is within the grasp of 
most young adults, large numbers have clearly failed to develop an ade- 
quate leve of literacy skills. The deficiency identified here is in young 
a ults skills at dealing with the more complex tasks embedded in print 
materials. Evidently, the printed word usually can be decoded, but the 
information ^obtained is not processed correctly to solve the problem. This 
is a key finding for programmatic efforts aimed at improvement, and one 
that is as significant for school curricula as it is for shaping adult literacy 
programs. It is also important in programs aimed at developing workplace 



12 





227 ' 



Occupations and Literacy 



I t would, of course, be desirable to know what level of literacy specific 
occupations require. This would involve' some systematic approach to 
looking at occupations and determining where they belong on the 
NAEP literacy scales. One method would be to use panels of employers to 
make determinations on a judgmental basis. Another would be to use the 
established tools of job analysis to systematically examine an occupation 
and determine its literacy requirements. 

So far, no research has been done that would permit confident state- 
ments about the literacy levels required for specific jobs; however ETS is 
currently conducting a feasibility study to determine how job analysis 
might be used to identify literacy requirements of five occupations. The 
results should be available this year. 

Still, some perspective can be gained by looking at the demonstrated lit- 
eracy proficiencies of people who are employed in a particular occupation. 
This, of course, begs the important question of what the occupation re- 
quires. Employers may have settled for less than they wanted; productiv- 
ity might be greater, and training and remedial education costs might fall, 
if literacy levels were higher. But despite these caveats, the method offers 
an advantage not to be ignored: the people to be assessed for proficiency 
are actually employed in the occupation; the market test has been applied 
We do know the occupations of the 21- through 25-year-olds in the 
Young Adult Literacy Assessment, and we can estimate the literacy pro- 
ficiencies of these young adults in a particular occupational category. But 
while this can be done with the literacy assessment, the estimates are nec- 
essarily limited by the sample size to broad occupational categories. These 

are managerial, professional, technical, sales, clerical, craft, operative and 
service. 

The sample of young adults contains many youth who are still in school 
full time, many who are both learning and earning, and many who have 
left education and are established in the occupational world. Of those who 
have left education, some did so as early as ages 16 to 18 when they 
dropped out or graduated from high school, while others just recently 
completed their college work. About one in four (26 percent) reported 
being enrolled in school at the time of the survey, and 69 percent of these 
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young adults were full-time students, while 31 percent attended class part 
time. 

Of course, many of the students, both full time and part time, were 
working and thus reported an occupation. However, these young adults 
have most likely not yet settled into entry-level positions in the occupa- 
tions that they are aiming for. To take an extreme example, a doctoral stu- 
dent may be working part time driving a taxicab; thus, his or her literacy 
level is not representative of that job's requirements. 

We are, however, able to separate the sample by the students' work 
status, so as to get near to the group that has completed the transition to 
work. Of the total sample, 54 percent — an estimated 9.6 million 21- to 25- 
year-olds— had been working full time for a full year. This group was 
largely settled into the work world, although some were attending school 
on a part-time basis. Table 1 shows the average (mean) performance of 
these full-time workers by occupational category in terms of the three lit- 
eracy scales. 



Table 1. — Mean proficiency scores on prose, document, and quantitative 
literacy tasks of full-time workers, age 21-25, by occupation: 1986 



Occupation Prose Document Quantitative 



Managerial 




308.3 ( 3 . 7 ) 

323.4 ( 5 . 4 ) 
311.2 ( 6 . 7 ) 
296.7 ( 4 . 8 ) 
301.1 ( 4 . 3 ) 

278.6 ( 5 . 7 ) 

281.5 ( 5 . 1 ) 

285.7 ( 5 . 2 ) 
277.0 ( 8 . 6 ) 


Professional 




Technical 




Sales 




Clerical 




Craft 




Operative 




Service 




Laborers 








NOTE: Numbers in parentheses are estimated standard errors. 



310.2 ( 6 . 8 ) 

312.1 ( 7 . 2 ) 
303.8 ( 8 . 0 ) 

298.1 ( 7 . 5 ) 

301.0 ( 3 . 5 ) 

281.2 ( 6 . 5 ) 
284.4 ( 5 . 2 ) 

292.0 ( 5 . 0 ) 

283.2 ( 9 . 2 ) 



, , S 9V^ CE: UnpubUsheddata from the National Assessment of Educational Progress, 
Adult Literacy Study. 1986. Educational Testing Service. ^ 



Young 



The reader may find it helpful to refer back to figures 1 through 3 for 
the meaning of various scale scores, in terms of the kinds of tasks these 
young adults can probably perform. The sample errors are such that small 
differences in scores, typically 5 or 6 points, should not be viewed as 
significant. (The general rule of thought is not to judge as significant those 
differences that are less than 2 standard errors apart.) 
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Ikble 2.— Percentages of young adults at or above selected points on the document literacy 
scale: 1986 
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What table 1 says is that the literacy levels of young adults reporting 
various occupations do differ considerably. On the document scale, the 
range is from 323 (professionals) down to 277 (laborers). Only 37.6 per- 
cent of all young adults in the literacy assessment were estimated tp be at 
or above the 325 level on the document scale. In contrast, 73 percent were 
at the 275 level or above, about where the average laborer scores. 

The distribution of mean proficiency scores for these full-time workers 
is similar on the other two scales, although the range is not quite so large. 
For prose literacy, the spread from high to low is 37 points, while for 
quantitative literacy it is 31 points. These compare to a spread of 45 points 
for document literacy. Because of the nature of the tasks on the document 
scale, we have considered it to be the most useful of the three scales for 
reflecting workplace literacy. Moreover, we see from this comparison that 
it does have the largest range in average proficiency among the occupa- 
tions. 

Table 2 shows the degree to which young adults and subgroups of 
young adults achieve these average document proficiency levels possessed 
by young full-time workers who have made the transition from school. 
While we will not summarize all the information in the table, we call 
attention to some disturbing facts. For example, more than 6 in 10 black 
young adults do not reach a proficiency equal to the average for "laborer," 
nor do almost half of Hispanic young adults. About 6 in 10 high school 
dropouts have not reached this level, and almost 8 in 10 are not at the 
average for clerical workers. In addition, half of high school graduates 
have failed to achieve the average proficiency of clerical and sales workers. 

Again, these are the average proficiencies of the young adults who re- 
ported being in these occupations at the time of the survey. We cannot 
say these are levels the employers thought were necessary, but they are 
the people who were hired. Also, given the young ages, these levels are 
more characteristic of entry-level workers in these occupations than of 
experienced workers. Finally, there is a large range of literacy proficiencies 
within any single occupation; a much finer classification of occupations 
would be necessary to establish a narrow range. Having stated these quali- 
fications, we do believe this kind of knowledge, on a more refined basis, 
would help illuminate the literacy competencies needed in the workplace. 
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The Fourth R: Work Force Readiness 



T he foundation for measuring prose, document, and quantitative lit- 
eracy has been laid with the conceptual work, measurement, and 
"scale anchoring" undertaken at ETS for the young adult 
literacy study, described above. The workplace needs the kinds of literacy 
measured on these new scales. But employers are asking for these skills 
and more. 

In addition to seeking workers who can deal successfully with print, 
employers have a list of skills and competencies they need now, and for 
which they project an increasing demand in the work force of the future. 
Many surveys of employers and reports of employer organizations have 
described these needs, but the National Alliance of Business characterized 
them most succinctly as the "Fourth R : Work Force Readiness." 

In the first half of the 1980s, a number of national. State, and local sur- 
veys examined what employers wanted entry-level workers, or high school 
graduates, to know and be able to do. Although these surveys were gen- 
erally carried out or sponsored by employer organizations, the questions 
were not always the same, and the specific words used often differed. 

Still, the central message came through loud and clear: there were several 
characteristics these new work force members needed to have, and 
employers were not seeing enough candidates who had them. A brief 
summary of the conclusions of the national-level studies will convey the 
messages employers have been sending. 

A Conference Board Survey of business executives found, for example, 
that while computer skills were on the rise, new workers' knowledge of 
basic skills was deteriorating. Indeed, 65 percent reported seeing declining 
abilities in reading, 74 percent in writing, and about 60 percent in mathe- 
matics. Current education reforms address these skill areas. However, 
these executives also reported that attitudes toward work and the workplace 
were a primary problem, and few were hiring recent high school grad- 
uates. 

The Committee for Economic Development surveyed a sample of For- 
tune 500 companies, as well as 6,000 small firms. Both the large and small 
companies agreed that the characteristics desired — and difficult to find in 
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young applicants-were " 8tr ! vi ^ ’ 

"priority setting/' "communicating, and worki g 

When Research and Forecasts \ I ^ C '^°^ye r t stiked character first in the 
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qualities looked for w en y With reg ard to education, 7 in 10 
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specifically designed *° ^XTrecoritended regular assessments both 
S d h.»^- readiness" and of employers 

needs. 
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In 1984, the National Academy Press published High Schools and the Chang- 
ing Workplace: The Employers ' View. Composed primarily of employers, this 
study group concluded that the critical core competencies "include the 
ability to read, write, reason, and compute; an understanding of American 
social and economic life; a knowledge of the basic principles of the phys- 
ical and biological sciences; experience with cooperation and conflict reso- 
lution in groups; and possession of attitudes and personal habits that make 
for a dependable, responsible, adaptable, and informed worker and citi- 

A it 



In 1987, the National Alliance of Business issued The Fourth R.- Work Force 
Readiness. Work force readiness, the report said, "includes thinking, reason- 
ing, analytical, creative, and problem-solving skills, and behaviors such as 
reliability, responsibility, and responsiveness to change." 

In Training America: Strategies for the Nation, the American Society for Train- 
ing and Development joined with the National Center for Education and 
the Economy to describe the skills workers will need if they are to partici- 
pate as full members of the autonomous work teams that employers are 
increasingly using in production. This 1989 report concluded that team 
members will need "high levels of interpersonal teamwork, negotiation, 
and organizational skills — skills that enhance group effectiveness — as well 
as leadership skills." 

Still another study. Workplace Basics: The Skills Employers Want, was issued 
by ASTD and the U.S. Department of Labor in 1988. This document 
summarized the basic skills in a very convincing manner, and we offer 
them as the best composite of all the messages contained in the decade's 
surveys and reports. 

3 R s, Reading, Writing, Computation. [We believe that from the 
employer's standpoint these are well captured in the three literacy 
scales described above — supplemented by assessment of writing.] 

Learning to learn 

Communication: Listening and Oral Communication 
Creative Thinking/Problem Solving 
Interpersonal/Negotiation/Teamwork 

Seif-Esteem/Goal Setting-Motivation/Personal and Career Develop- 
ment 

Organizational Effectiveness/Leadership 



19 



ERIC 



323 BEST COPY AVAILABLE 



234 



i 

When the skill needs of the workplace are discussed in the media, the 
attention is usually wholly focused on formal education or on literacy. 
These are. very important considerations, not to be taken lightly or 
downplayed. But they are just one component of the ability required to 
function in the modem workplace. If a young person can deal well with 
print, but does not listen carefully to instructions, lacks personal dis- 
cipline, and fails to function effectively in group efforts, then most large 
employers believe that there will be a deficiency in workplace functioning. 

What vye argue, then, is that in addition to giving attention to prose 
documentj *** quantitative literacy, this expanded list of skills and 
characteristics must be addressed. These skills have not become part of 
any large-pcale assessments of students and young adults, so nothing is 
yet known about the extent to which new workers are aware of their rel- 
ative importance, let alone possess them. Also, little exists in the way of 
knowledge regarding whether and to what extent schools emphasize these 
abilities. And, finally, there are significant questions regarding how skills 
such as teamwork can be assessed. Indeed, careful development work 
would be required to field such an expanded assessment. Still, we do 
know that : employers frequently find these skills missing in young people 
applying for jobs. 
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Literacy in the Future Workplace 



T he typical discussion of workplace literacy needs dwells first on cur- 
rent employer experience; it moves quickly to an exposition of the 
rapid advancement in literacy requirements, and then predicts 
a growing mismatch between skills workers have and those the workplace 
needs. We are comfortable with the proposition that there now exists a 
large mismatch between proficiencies and needs; we are much less so with 
projections that, based on occupational requirements, claim a growing mis- 
match in the future. 

Looking at the post-World War II period as a whole, it can be said with 
some confidence (although it would take the writing of a book to firmly 
establish it) that there has been a gradual "creeping up" of the education 
and literacy requirements of jobs. But it is hard to find evidence that there 
has been any sudden or rapid escalation in those requirements. It is clear, 
for example, that in each decade more people in the same occupational 
classifications have possessed more education, but it is not known how 
much of that gain has been due to steady advances in school completion 
at the various levels, and how much to changes in the jobs themselves. 
Another confounding fact is that a market system results in employers 
seeking the best prepared labor supply that the supply and demand situa- 
tion affords them. Still, they will often settle for lesser skills when work- 
ers are in short supply. One example is military recruitment: when supply 
decreases, the military is forced to choose among lower ability levels to 
meet manpower requirements. Technological advances are also thought by 
a great many to raise educational requirements; at the same time, however, 
technology has been widely used to replace skilled labor, as well as to 
eliminate people and jobs entirely. This is how many productivity in- 
creases have been achieved. 

One of us (Barton), when asked to predict the future, is on record as 
saying (in 1986) that 

We simply do not know how technological changes in methods of 
production and service delivery, organizational changes in the 
management of production, creation of new products, and thorough 
advertisement, the creation of an effective demand for them, and 
changes in the international market, will shape education require- 
ments in the future. Focus on the dramatic factors that raise edu- 
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cation requirements may be overshadowed by more mundane 
developments. For example, at Christmas of 1983, the phenomenal 
sale of Cabbage Patch Dolls was largely a media feat— they re- 
quired no new skills to manufacture. Tens of thousands of young 
people have lost jobs pumping gas because of the relatively new 
service stations" where only one person is required to take money. 
A raft of new low-skilled jobs were created — in the establishment 
of inspection stations in airports. . 

In 1987, the evidence was sifted by the Panel on Technology and 
Employment's Committee on Science, Engineering, and Public Policy, and 
reported in Technology and Employment, published by the National Academy 
Press. The conclusion: "Like the literature and evidence on the employ- 
ment impacts of technological change, the empirical evidence of tech- 
nology s effects on skills is too fragmentary and mixed to support con- 
fident predictions of aggregate skill impacts. Despite this uncertainty, 
however, the evidence suggests that the skill requirements for entry into 
future jobs will not be radically upgraded from those of current jobs." 

A starting point in any effort to see into the future of educational 
requirements is to be able to project the occupations and the numbers em- 
ployed in them, and that itself rests on prior work of constructing an 
"input-output matrix" of the U.S. economy. From these projections, some 
idea of educational requirements can be gleaned, although difficulties 
remain. (For example, occupations with the same name can involve 
changed job content.) While the U.S. Department of Labor's Bureau of 
Labor Statistics (BLS) has a pretty good record in making these projec- 
tions, they remain approximations of the future. Harold Goldstein, for- 
merly an associate commissioner of BLS responsible for projections, has 
evaluated their accuracy. The results were as follows: 



1960-75 1970-80 



On target 39.5% 32.8% 

Close 39.5% 26.8% 

Not close 21.1% 40.6% 



That the earlier efforts were more on target than the later ones is not 
encouraging, although BLS has made many improvements in the estimat- 
ing methodology since the 1980 projections. 
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Obviously, these occupational projections are frequently the basis for 
statements about what the future holds, and the fastest growing occupations 
are pointed to as having higher educational requirements. In the BLS 
projections for the period 1988 to 2000, the 10 fastest growing occupations 
are: 



Paralegal personnel + 75% 

Medical assistants + 70% 

Home health aides + 68% 

Radiological technologists and technicians + 66% 

Data processing equipment repairers +61% 

Medical records technicians + 60% 

Medical secretaries + 58% 

Physical therapists + 57% 

Surgical technologists + 56% 

Operations research analysts + 55% 



While these occupations conjure high educational requirements, and 
most require postsecondary degrees, they account for a job growth of only 
694,000, out of a projected total job growth of 18 million during the last 
12 years of this century. 

In terms of numbers of jobs projected, a quite different picture emerges. 
The top 10 occupations in terms of absolute job growth are: 



Salesperson, retail + 730,000 

Registered nurses + 613,000 

Janitors and cleaners + 556,000 

Waiters and waitresses + 551,000 

General managers and top executives + 479,000 

General office clerks + 455,000 

Secretaries except legal and medical + 385,000 

Nursing aides and orderlies + 378,000 

Truck drivers + 369,000 

Receptionists and information clerks + 331,000 



These 10 occupations account for almost 5 million projected new jobs, 
and from 6 to 8 of the occupations are not intuitively associated with 
increasing literacy requirements, although the literacy standards employers 
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have for current openings may often exceed skills of entry workers. Fol- 
lowing these 10, in the order of growth, are cashiers, guards, computer 
programmers, food counter workers, and food preparation workers. Taking 
these occupations apart and forecasting their educational requirements is 
no easy task. 

Recently, the Hudson Institute, in its report Work Force 2000 , attempted 
to estimate the educational requirements of jobs in the year 2000, by com- 
bining the BLS occupational projections with the scales of educational 
requirements used in the Dictionary of Occupational Titles , published by the 
U.S. Employment Service. The Hudson Institute reported that the average 
rating of language requirements for current jobs was 3.1 on a scale of from 
1 to 6. The "fast growing" occupations were rated at 3.8, the "slowly 
growing," 2.7, and the "declining," 1.9. The report concluded that "the 
fastest growing jobs require much higher math, language, and reasoning 
capabilities than current jobs, while slowly growing jobs require less." 

These numbers suggest some shift toward higher requirements. But we 
don't know from this report exactly how "fastest growing" jobs are tab- 
ulated; we have seen from the BLS projections that occupations with a 
high percentage growth may be on a small base, and the numbers of jobs in 
such categories relatively small. In the separately available appendix to 
Work Force 2000 , a table provides scale ratings for all jobs in 1984 and all 
jobs projected for the year 2000; the language scale score for 1984 is 3.1, 
compared with 3.2 for 2000. The difference is well within the margin of 
estimating error. Nevertheless, the Work Force 2000 report has pioneered in 
estimating future educational requirements, and this marriage of edu- 
cational requirements with occupational projections is a very promising 
approach and is to be encouraged. 

The job openings in the year 2000 will be in all occupational categories, 
not just in the net increase in jobs in the fastest growing occupations; 
there will also, of course, be positions available in those occupations 
registering a net decline in average employment. There is a constant mill- 
ing about in the labor force. Between 1988 and 2000, the BLS estimates 
that 43 million people will enter the labor force and that, taking account 
of labor-force withdrawals, the net number of new entries will be 19 mil- 
lion people. No estimate has been made of the educational requirements 
that those who enter jobs in the year 2000 will be expected to meet. 

One thoughtful comment about change was made in the National Acad- 
emy of Science report High Schools and the Changing Workplace: 



24 



ERIC 




239 



The pace of change affecting entry-level jobs will not differ much 
from that of the past but, as graduates mature and move out of 
entry-level jobs, the effects of change may become more pro- 
nounced. [The many changes] imply a continuous need to under- 
stand and master constant technical change. 

From the worker s standpoint, the nature of many jobs will change 
in the coming years, although their titles will remain the same. 
Obviously, the degree of change will vary from job to job and in- 
dustry to industry, sometimes drastically, sometimes slightly. 

A National Research Council report. Education for Tomorrow's Jobs, offered 
this assessment: 

There is no agreement of what these skill requirements will be or 
even in the general direction of change. 

The precise direction this change will take is even more difficult to 
specify. . . . Given the uncertainty regarding the skill requirements 
of the economy, it is essential that the education of America's 
young people be designed to enhance their abilities to adapt as nec- 
essary to these changing requirements. 

Two researchers who have written throughout the 1980 s on the edu- 
cational requirements of jobs are Russell W. Rumberger and Henry M. 
Levin. They have just published a reanalysis, entitled Schooling for the Modem 
Workplace. The principal conclusion was that: 

The average educational requirements of future jobs will not be 
significantly different than current jobs, as both high-skilled and 
low-skilled jobs will continue to exist in the future economy. 

This discussion by no means is meant to suggest that the Nation should 
be less vigilant about raising literacy levels. The evidence is to the con- 
trai T present literacy levels are much too low to meet current needs and expectations. 
Further, there is no reason to expect needs to diminish, even if they ad- 
vance slowly or not at all. From the labor supply standpoint, we face a 
future where minority populations with traditionally lower educational 
attainments and traditionally lower literacy levels will be a growing 
proportion of new labor force entrants. Thus, society will have to run 
faster just to stay in the same place. 

Moreover, we do counsel against failing to press onward in efforts to 
predict the future occupational structure and its underlying educational 
requirements. 
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Some Implications and Recommendations 



1. Addressing Inadequate Literacy 



W e have stated, and reiterated, the distinctions between the 

common approach of "counting the illiterates" and the more 
sophisticated strategy of profiling the population on literacy 
proficiency scales. We have emphasized this distinction for two reasons. 
First, literacy is a complex set of socially practiced knowledge and skills 
that needs to be understood in terms of what workplace, doing what kinds 
of tasks, and delivered in what kinds of training and education programs. If 
the problem is not framed correctly, the answer derived will be the wrong 
one. 



The second reason is that the nature of the problem cannot be measured 
if the population is sorted into just two groups. What we have seen is that 
most young adults, high school graduates, and, to a lesser extent, high 
school dropouts can perform simple tasks conveyed through the printed 
word. Meanwhile, most illiteracy programs seem aimed at teaching adults 
to read the printed word. But while there are significant numbers of adults 
who can benefit from this type of training, they are not our major dif- 
ficulty. Rather the larger problem, in terms of numbers, are those who can 
read at some basic level, but seem to lack the necessary information- 
processing skills that would enable them to carry out the kind of mod- 
erately complex tasks frequently encountered in everyday life. 

These are the facts with which workplace literacy campaigns must come 
to gnps. How are these information-processing skills to be improved? Are 
the programs we are creating directly addressing them? Such discussion 
raises, in turn, the important question of whether better ways to teach the 
current curriculum are needed or whether the need is to rethink not only 
how schools teach but what is taught. 



2. Expanding Measures of Proficiency in Literacy 
Tasks Beyond Classroom Reading 

l 

One of the problems employers and others concerned with literacy 
encounter is that the existing instruments available to assess literacy are 
geared to measuring what students can do in school classroom-type read- 



27 



O 

ERJC 



33 $ 



242 



ing exercises. The kinds of instruments ETS-NAEP used in the young 
adult literacy assessment (the results of which are described above) are not 
currently available for testing individuals. Such instruments would, how- 
ever, give a truer picture of the skills entry workers have, and permit more 
appropriate placement in remedial programs. ETS is currently under con- 
tract to create such a test for clients of several U.S. Department of Labor 
programs — Unemployment Insurance, the Employment Service, and the 
Job Training Partnership Act/ Not only will this secure test be available 
for public use by summer 1991, but ETS is also under contract to produce 
a nonsecure test of adult literacy for Workplace Resources, a division of 
Simon & Schuster, Inc. 

With measurement instruments that identify the level of literacy attain- 
ment and that are linked with instruction geared to raising workers and 
potential workers from where they are now, workplace competencies can 
be increased. New measures, however, are only one step; people who di- 
agnose literacy needs and develop and operate literacy programs need to 
understand the distinction between classroom reading proficiency and real 
life literacy requirements and gear programs to raising literacy levels. 

3. The Future is Now 

We have concluded that there is now a very large gap between the lit- 
eracy skills possessed by a large proportion of young adults and the needs 
of the workplace. With the proper approach, this gap can be addressed 
now; the problem is upon us. Also, we do see an increasing problem in a 
future that can be foretold: a growing percentage of new labor force en- 
trants will be from minority groups whose educational performance has 
traditionally lagged behind that of others. If this gap is not closed soon, 
society will have to run faster just to stay in the same place, since those 
youth with lower literacy levels will become an increasing proportion of 
the total. 

At the same time, we are not comfortable resting the case with projec- 
tions that conclude, as so many have recently done, that the educational 
requirements of jobs are advancing rapidly. In any event, the Nation does 
not have the luxury of waiting until a future problem develops, since an 
urgent need exists for action now. 

We have said that there will be change, and this is particularly true in 
how employment organizations are structured; we will see more coopera- 

*The contract also provides for profiling the literacy proficiency of these three populations. 
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tive effort and greater reliance on worker initiative. This will affect both 
younger and older workers, and will require that they be able to adapt. 
Otherwise, their worklife transitions will be rocky, and industrial 
productivity will suffer. Barton commented on these trends, already well 
evident, in 1982: "As workers are expected to participate more in problem 
solving and decision making, a greater premium will be placed on broad 
preparation as compared to narrower skill training. I would speculate that 
the need will be for more teaching of problem-solving skills, teamwork 
methods, and knowledge of how organizations function." 

4. Improving Literacy Skills in the Schools 

To raise literacy as it is defined here, instructional practices must be 
broadened. Although the two are obviously related, a distinction has now 
been established between the knowledge and skills associated with reading 
in school settings and those associated with performing tasks frequently 
encountered in adult contexts outside the schools. Despite many decades 
of theory and research on how young people learn to read and how read- 
ing is best taught, the book is not closed on the subject of acquiring lit- 
eracy. Using the new literacy scales, we need to assess literaq/ in the 
schools, beginning in middle school, if we are to do better in aiding its 
development. 

There are good reasons to believe that traditional classroom approaches 
fall short in teaching the kind of problem-solving tasks used in the lit- 
eracy assessment. Students may need to be put into problem-solving situa- 
tions that give them greater opportunity to use and apply these skills and 
strategies — or at least be provided with classroom simulations of such 
experience. This was the view expressed in the recent report Training Amer- 
ica: Strategies for the Nation , issued by the National Center on Education and 
the Economy and the ASTD: 

Employers have long been advocates of an applied pedagogy. They 
argue that learning that occurs in some functional context produces 
better students as well as better employees. 

This view gains support from the research described in Sylvia Scribner's 
and Joy Stevens' 1989 report. Experimental Studies on the Relationship of School 
Math and Work Math. They conclude: "It appears from the high level of 
literality observed in our studies, that school math instruction does not 
promote the use of expert problem-solving strategies in nonschool situa- 
tions. This observation is in keeping with judgments reached by a number 
of educators on the basis of student math performance in school and test 
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situations/' What they suggest is "to situate some aspects of math instruc- 
tion in contexts of actual practice." They are not the first to caution 
against divorcing the classroom from the rest of life: "As societies become 
more complex in structure and resources, the need for formal or inten- 
tional teaching and learning increases. As formal teaching and training 
grow in extent, there is danger of creating an undesirable split between the 
experience gained in more direct associations and what is required in 
school." What John Dewey said a long time ago is being proven by sci- 
entists today. 

We urge the development of more instructional approaches that involve 
or simulate real experience. 

5. Improving Measures and Tools for Evaluation 

An Employment Readiness Profile 

While literacy skills are of great importance to employers and 
productivity, employers have made it clear that they have requirements 
beyond literacy and that the characteristics they seek are in short supply. 
(These "employment readiness" characteristics are reviewed above in the 
section "The Fourth R: Work Force Readiness.") We believe that useful 
assessment instruments can be developed to track national progress in all 
of these areas. Such an effort should be carried out with the participation 
of employers and employer organizations, as well as labor market experts 
and educators. We have called this an Employment Readiness Profile; it 
could be administered periodically to national samples of school leavers, to 
help track national progress in developing a work force that can meet 
coming competitive challenges. 

Tracking Literacy: Comparable Measures Over Time 

Literacy experts are continually asked: Is there more or less literacy than 
10 or 20 years ago? While there are grounds for making guesses, the truth 
is unknown. Literacy measurement has been sporadic; it has used different 
definitions and concepts; and it has been applied to different age groups. 
One measure of concern about literacy is willingness to periodically meas- 
ure it in a consistent manner. A start has been made with the upcoming 
National Adult Literacy Survey that ETS will conduct under contract with 
the U.S. Department of Education. Not only will it survey all adults ages 
16-64, but it will commence to track trends by permitting comparisons be- 
tween the 21- to 25-year-olds of 1986 and those of 1992. These popu- 
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lations will also be compared with enrollees in Job Training Partnership 
Act programs. Employment Service applicants, and Unemployment Insur- 
ance claimants assessed in 1990. 

Developing Literacy Tasks From Workplace Materials 

While we believe that the conceptual and scaling approaches used for 
the young adult literacy assessment, with its prose, document, and quan- 
titative literacy scales, are appropriate for workplace use, additional re- 
sources for developing assessment items could increase their usefulness. 

While literacy would still be reported on the same scales as the NAEP 
assessment, these additional tasks — assessment items identified with work- 
place settings — could be developed with the aid of employers. For exam- 
ple, the prose tasks could use print found in a variety of work settings; the 
documents would be kinds frequently encountered in employment, and so 
would the quantitative tasks. However, while the tasks would be drawn 
from workplace settings, they would have to be of general usefulness in 
simulation exercises. They could not be so dependent upon context as to 
require specialized knowledge on the part of the test taker. If items could 
not be developed to meet this criteria, such an effort would not be fea- 
sible. 

Analysis of Occupations 

Much progress has been made in measuring literacy proficiencies. A lot 
more will occur when the new National Adult Literacy Survey is com- 
pleted. But more knowledge is needed about the literacy levels that dif- 
ferent occupations require, particularly the typical entry-level occupations. 

We have suggested one line of analysis in showing the actual literacy 
levels of the young adults in particular occupations in the 1986 study. ETS 
is also using job analysis (called literacy audits) in a limited feasibility 
study of five occupations. Expert judgment by panels of employers is an- 
other approach. 
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By the nature of this assignment, we have dwelt on the literacy needs of 
the workplace. And to some extent, we have contrasted workplace tasks 
with those of the classroom. By making a sometimes arbitrary distinction 
between the workplace and the school, we do not want to contribute to 
widening the chasm which has existed, and is now the source of some of 
t e growing consensus about the inadequacy of literacy skills. 



The larger point we would close with is that to envision distinct, sepa- 
rable kinds of literacy that are called upon in isolation one from another is 
not a very useful concept. Specialized literacy skills are not things that can 
be turned on and off in different settings. Life is not so compartmen- 
talized; it is, in fact, a seamless web. 



AH citizens need a broad base of literacy skHls to function in the school 
m the family, in the community, in the voting booth, and in the work- 
place. The chaUenge is to understand how such skUls and knowledge in 
these several contexts are similar, and how they are different, as part of a 
critical effort to find ways to promote their development. 



Our desire has been to look at literacy through the window of the 
workplace, but as one window in a house with many rooms. Through this 
window, we see prose, document, and quantitative tasks that occur with 
some regularity in our society. More importantly, for too many young 
adults in our society, these tasks represent competencies that are not being 
acquired. Unless something is done, the percentage of young people who 
demonstrate these deficiencies is likely to grow over the next decade. As a 

Nation we will see these deficiencies impact greatly on our social and eco- 
nomic fabric. 
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Preface 



The William H. Angoff Lecture Series Reports are the most recent addition to the roster of Policy 
Information Center publications. 

In the 10 years since the National Council of Teachers of Mathematics (NCTM) published the oft 
cited Curriculum and Evaluation Standards for School Mathematics, much of the discussion and debate in 
educational reform has stemmed from one word: standards. While professional and product standards are 
widely accepted, educational standards are still in their infancy. Educators, parents, and policymakers across 
the U.S. are struggling to achieve consensus on what schools should teach and what students should know 
and be able to do. The quest for meaningful standards can be seen everywhere — from the burgeoning 
documents from various professional associations following suit with the NCTM to state-level initiatives 
in Kentucky and California to the national efforts to establish skills standards. 

In the current report, Dr. Bert F. Green, professor of psychology at Johns Hopkins University, turns 
our attention to the key issues associated with the daunting task of setting performance standards. Dr. 
Green observes that standards are poorly understood and that the measurement community faces numer- 
ous challenges in identifying methods for standard setting. However, he argues that the primary policy 
problem is not how to set standards, but how many standards should be set, and on what measures should 
standards be set. 

Drawing on the rich experiences of his work in the fields of psychometrics, statistics, cognitive 
psychology, artificial intelligence, and computerized adaptive testing and insights from his work on 
numerous advisory panels and professional associations, such as the National Assessment of Educational 
Progress and the American Psychological Association, Dr. Green examines the measurement issues of 
standard setting in terms of the broader purposes that standards serve. 

I would like to thank the following individuals for their contribution to this publication: Ric Bruce 
designed the report; Carla Cooper provided desktop publishing services; Jim Chewning coordinated pro- 
duction; and Shilpi Niyogi was the editor. 



Paul Barton 

Director, Policy Information Center 



Preamble 



I want to thank Henry Braun, and Educational Testing Service, for inviting me to deliver the 
annual Angoff Memorial Lecture. Choosing a relevant topic was not difficult. Almost any topic in educa- 
tional measurement would reflect Bill Angoff's work, since he had a hand in so many of the technical 
problems of measurement. Indeed, most topics in psychology or education would be appropriate because of 
Bill's service and devotion to the American Psychological Association and the science it represents. Bill and 
I served together on the APA Council of Representatives, as representatives of the Division of Evaluation, 
Measurement, and Statistics. We often shook our heads at some of the antics of our colleagues. Despite our 
scientific conservatism, we shared an interest in developing techniques to address the new problems that 
measurement always faces. 



O 

ERIC 




5 



343 



256 



Introduction 



/ 

JLn the inaugural Angoff memorial lecture. Bob 
Linn (1995) spoke about some aspects of educational 
standards. Today I want to talk with you about set- 
ting performance standards, an arena to which Bill 
Angoff contributed more than he expected. It is an 
area that various constituencies see quite differently, 
and one that causes heartburn among measurement 
experts. I shall also comment on content standards, 
and the tenuous link between content standards and 
performance standards, an arena that has scarcely 
been addressed by psychometricians. I have not con- 
tributed directly to the study of standards, but have 
been observing it closely, as a technical advisor to 
the National Assessment of Educational Progress, and 
as a technical advisor to the Maryland State Perfor- 
mance Assessment Project. I learned a lot at a joint 
conference on large scale assessments in 1994, espe- 
cially from Michael Zieky, Samuel Livingston, and 
Samuel Messick. 

Educational standards are always of concern 
to educational policymakers. There is perennial pres- 
sure for improvement. Colleges complain that the 
incoming students are not well-prepared. Employ- 
ers complain that some high school graduates have 
only a slim grasp of the basics. In the 1970's, there 
was an upsurge of interest in doing something about 
standards. Most people felt that no one should be 
given a high school diploma without exhibiting at 
least minimum competency in the basic subjects of 
reading writing, math, and perhaps social studies. The 
minimum competency movement swept the coun- 
try. (Jaeger, 1989). Tests of the basic skills were 
devised, and given to high school seniors. Cut points, 
or minimum competency standards were placed on 



the scales. To many, the demands seemed minimal 
indeed. Nevertheless, some students failed. 

At first, the onus was placed on the students. 
Then students and their families complained that 
they had not been given an adequate opportunity to 
learn the required skills. Some states, such as Florida, 
had introduced the standards abruptly, rather than 
phasing them in over a span of several years. The 
court permitted Florida to go ahead, while the edu- 
cation vocabulary gained a new phrase and acronym, 
Opportunity To Learn (OTL) (Citron, 1983). At 
present, minimum competency tests are widely used 
and accepted. In Maryland, the tests are given as 
computer-based adaptive tests, and many students 
pass them before they enter high school, although 
the tests are not required until high school 
graduation. 

Policymakers felt that the minimum 
competency tests would stimulate teachers and learn- 
ers alike. The policy may have worked. The 
performance of the worst students was improved, 
without noticeable harm to the better students. Still, 
as Linn (1995) noted, there was general agreement 
that the minimum competency criterion led to more 
emphasis on the basics, and less emphasis on more 
advanced topics in the curriculum. 

High Stakes Assessments 

A new round of concern developed in 
the 1980's. John Cannell (1988) reported that stan- 
dardized tests of achievement showed that most 
states were above average. Cannell named the effect 
after Garrison Keillor's fictional Lake Wobegon, 
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where all the men are strong, all the women are good- 
looking, and all the children are above average. 

Alas, we can't all be above average. A variety 
of problems were identified, including out-of-date 
norms, teachers teaching to the tests, and in some 
cases teaching the actual test items themselves. It 
began to dawn on educational policymakers that 
when stakes are high, any empirical evaluative cri- 
teria, like test scores, are vulnerable to manipulation. 
Sometimes, this concept is slow to be recognized. In 
the State of Maryland, nearly all students in the 
public schools were being promoted every year. The 
phenomenon was not the result of remarkable edu- 
cation, nor above-average students, but seems to stem 
from the practice of allotting state money to school 
districts on the basis of the promotion rate in the 
schools. That formula has recently been changed. 

Just last month, the Maryland State Depart- 
ment of Education reported that their annual Per- 
formance Assessment in the elementary schools had 
been compromised. Baltimore schools' Student 
Assessment Service published a teachers' guide to 
the assessment that included a few of the actual ques- 
tions from the forthcoming assessment, with a slight 
change of wording. Dr. Steve Ferrara, whose State 
office was responsible for the assessment called it a 
significant breach of security. Dr. Amprey, Superin- 
tendent of Baltimore Schools, called it "poor judg- 
ment." ( The Baltimore Sun. October 21, 1995.) Simi- 
lar kinds of problems have plagued educators across 
the country since the assessment stakes have been 
raised. 

The current wave of excitement about 
improving the quality of education is built to some 
extent on the notion that, "If you can't beat them, 
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join them. The strategy seems to be to build a test 
that represents what the students should know, so 
that teaching to the test becomes teaching the cur- 
riculum that is central to student achievement. If 
standards are set on relevant assessments, the teach- 
ers will scramble to prepare students to do well on 
the assessments. 

Criterion-Referenced Test Scores 

Cannell's jibe about too many above average 
students called attention to the fact that a norma- 
tive scale is not really what is needed in assessing 
achievement. Norm-referenced tests tell us whether 
Susie from a rich suburban school knows more than 
Johnny from a poor city school. That is not really 
the point. What we really want to know is "What do 
the students know and what can they do?" This kind 
of question implies an absolute scale, rather than a 
relative scale. In psychometrics, such a scale is called 
criterion-referenced (Berk, 1976). Achievement 
assessment calls for criterion-referenced scales. In 
fact, these names, norm-referenced and criterion-ref- 
erenced, are not totally appropriate. The difference 
between the two is not only in the referents for the 
test scales (distributions of scores for norm refer- 
ence vs. content coverage for criterion reference) but 
in the test construction itself — performance differ- 
entiation is the main factor driving the construction 
of norm-referenced tests, whereas content coverage 
is the main factor driving the construction of crite- 
rion-referenced tests. 

It sometimes seems that policymakers do not 
understand the distinction between criterion-refer- 
enced and norm-referenced tests. In the good old 
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tradition of competition, the Secretary of Education 
prepared a wall chart showing, among other things, 
average SAT scores for each of several states. Tech- 
nically, this is an outrageous misuse of SAT scores 
(Wainer, Holland, Swinton & Wang, 1985; Wainer, 
1986). The SAT is designed to be norm-referenced. 
Its purpose is to differentiate among students bound 
for those colleges that require SAT scores. The SAT 
is in no sense an achievement test, even though it 
measures verbal and quantitative abilities that have 
been developed and expanded in school. 

The National Assessment of Educational 
Progress (NAEP) improved upon the wall chart by 
conducting a state-by-state assessment of mathemat- 
ics achievement, using a criterion-referenced mea- 
sure. This provided a more reasonable way of 
obtaining comparisons between states. Still, compar- 
ing achievement in one state with that in another 
state does not seem to be very enlightening. States 
should be interested in what their students know, 
not in whether their students know more than the 
students in other states. The recent rash of interna- 
tional assessments seems likewise designed to show 
that the United States students are not number one. 
At the local level, school administrators tell me that 
they observe great interest in school improvement 
on the part of real estate agents, who are probably 
seeking a comparative advantage. 



Elements of Performance 
Standards 



JL he term "standards" is used in so many ways 
in education and testing that it sometimes seems that 
we aren't even talking about the same concept. In 
order to sort out this melange, it may be helpful first 
to consider the elements that go into a performance 
standard, and then to consider how those elements 
apply to performance standards that are used for dif- 
ferent purposes. The discussion will be confined to 
standards applied to people, or groups of people, 
rather than standards for products like electric wir- 
ing, or vacuum cleaners. 

In setting performance standards for stu- 
dents, one must first have a scale on which to set it. 
Perhaps standards could be set without a scale, but 
that seldom happens in education. The only alterna- 
tive is a list. Standards for products are generally 
lists. One such list is the Standards for Educational 
and Psychological Testing and Assessment , prepared 
jointly by the American Educational Research Asso- 
ciation, the American Psychological Association, and 
the National Council on Measurement in Education 
(1985). Tests are expected to live up to each one of 
those standards. Sometimes shortcomings in one area 
might be ignored because of good performance in 
other areas, but that is not the intent of the stan- 
dards. There are, of course, no sanctions for tests that 
don't meet the standards. 

But, except for lists, performance standards 
are set on score scales. The choice of the test or tests 
is critical. Is the test to be built mainly for differen- 
tiation, or is it to be a criterion-referenced test, built 
mainly to represent the content domain? Nearly 
always, a criterion-referenced test is needed. There 
is then the question of how to specify the content 
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that the test is to cover. This involves content stan- 
dards, which will be considered later. 

Next, is there to be one standard on a scale, 
as in minimum competency, or should other levels 
of competency be recognized? That is, should the 
grade be pass-fail, or should it be A, B, C, D. There 
are many circumstances in which several levels 
would be useful. 

Is one scale enough, or is there so much 
variety of content that several scales should be used? 
If there are several scales, is the standard to be an 
overall standard, obtained from some combination of 
scale scores, or should there be a standard on each 
scale, and some rule for an overall decision. In high 
school tests of minimum competency, for example, 
there is a minimum standard for each of several key 
subjects. The student usually must pass each subject. 

Finally, how shall the cut-point actually be 
determined? There are some sophisticated methods 
for setting standards, the most popular of which is 
the so-called Angoff method. The term "so-called'' 
is appropriate because Bill described that method off- 
handedly, and mainly in a footnote, in the classic. 
Scales, Norms, and Equivalent Scores (Angoff, 1971). 
In this procedure, each of a set of expert judges is 
asked to imagine a person whose skill is just at the 
borderline between acceptable and unacceptable, and 
then to mark the exam as that person would have 
marked it. The average score of the papers marked 
by all the experts is then taken as the cut-point or 
performance standard. The main problem is getting 
judges to agree about what is meant by a person at 
the borderline. 

The Angoff method has some variants. Each 
judge can be asked to indicate, for each test item, how 
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likely it is that the borderline person would give the 
correct answer. Critics say that this amounts to ask- 
ing people to judge probabilities, a task that they can't 
do very well. But the responses can be rescaled for 
maximum interjudge agreement; the actual prob- 
abilities don't have to be believed for the method to 
work. 

Instead of imagining a marginally competent 
person, Michael Zieky and Samuel Livingston of ETS 
point out that it would also be possible to assemble a 
group of real people who were marginally compe- 
tent, and to find out how they did on the test. 
Another scheme is called the contrasting group 
method. A group of well qualified professionals, and 
a group of aspirants who are clearly not qualified, 
are both given the exam. The cut point is set so as to 
best discriminate between the two groups. 

There are also some not-so-sophisticated 
methods in regular use. For example, norms are 
sometimes used as standards. That may seem arbi- 
trary, but it happens. Course grades, for example. 
Course grades in school and college are a species of 
performance standard. Teachers sometimes grade on 
the curve (norm referenced), or else they assign 
grades based on their own and their schools' stan- 
dards (criterion-referenced). It is not always easy to 
know what grades mean. I shall always remember a 
Shakespeare course I took at Yale, when I briefly 
thought I would major in English. I did not under- 
stand very much of the seminar discussion, but I did 
OK on the biweekly papers, or so I thought until I 
showed one of my graded papers to a classmate who 
exclaimed, "B+ ! That's the lowest grade anyone has 
gotten on a paper this semester." I quickly converted 



to a math major. 

10 

? 



261 



Purposes Of Performance 
Standards 



The choices for the elements of a particular stan- 
dard depend to a large extent on the purposes that stan- 
dards are intended to serve. They can be used in certifi- 
cation :, such as when minimum competency tests are used 
to certify a high school graduate. Standards can also be 
used as predictors , such as when standards are set on 
entrance exams for college or for employment. Standards 
may serve merely as descriptors. And finally, standards 
can be used as motivators. 



Standards as Certifiers 

competency tests and the many 
professional certification exams — like the actuarial 
exams, the certified public accountant exams, bar 
exams for attorneys, and medical board exams — are 
designed to establish competence. These tests are 
intended to be criterion-referenced, in the sense that 
their main purpose is to represent the relevant con- 
tent. There might also some consideration of testing 
method. An architect should be asked to design some- 
thing. A surgeon would seem to need to display 
actual hands-on performance, as well as job knowl- 
edge, but perhaps a portfolio of patients could be 
submitted. 

One cut-point would seem enough. A person 
is, or is not, certifiable, as in the case of a minimum- 
competency exam. But often several scales are 
involved. The high school student must demonstrate 
competence in several areas. Almost all professional 
certification exams involve tests in several areas. The 




main question for certification examinations is how 
many different scales are to be assessed, and how 
the results are to be combined. Sometimes the dif- 
ferent areas are sufficiently correlated that a single 
overall score can be obtained by some kind of 
weighted or unweighted average. A history achieve- 
ment test might include American history and world 
history in some proportion, and vast knowledge of 
one can compensate for little knowledge of the other. 

More often, separate standards are set on each scale. 

A C.P.A. needs to be adequately adept at business 
law and ethics, auditing, business accounting, and 
non-business accounting. Each of four exams must 
be passed, within three years. The actuaries had, 
when I started taking the series, eight exams, which 
all had to be passed, but over an extended time span. 

In practice, certification standards are often 
set without benefit of psychometrics. One certifica- 
tion exam has a mandated passing grade of 70% of 
the items. However, adjustments are made so that 
30% of the candidates pass. Similar methods are used 
in a number of specialty exams. Passing grades on 
bar exams are frequently set by this percentage 
method, but the percentage varies a little, from year 
to year, depending on how many new lawyers are 
needed that year. 

Standards as Predictors 

Standards are often used for college entrance 
or employment tests. For example, a college academic 
department might believe that a student who scores 
less than 500 on any section of the Graduate Record 
Examination General Test is unlikely to make the 
grade in graduate school. In such situations, it is 
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important to set the standard on the right scale. 
Sometimes the Angoff method is used to set cut-offs 
on the predictor of success, the test. Judges are asked 
to estimate the probability that a successful worker, 
or a successful student, would correctly answer each 
item on the selection test. That is an incorrect appli- 
cation of the methodology. The judgments are on the 
wrong continuum. 

In a prediction situation, the standard should 
first be set on the criterion. When predicting job per- 
formance, it is job performance that should be 
dichotomized. In predicting academic success in col- 
lege, it is college grade 
point average (g.p.a.) on 
which standards should 
be set. First we locate the 
criterion point that 
separates the successes 
from the failures, then 
we need to locate the cut 
point, i.e., the value that 
predicts that criterion 
cut point. For example, 
the cut point on the pre- 
dictor could simply be 
the score for which the 
person is predicted to 
have a 50-50 chance of 
success. Figure 1 depicts 
this situation. It is 
important to notice that 
when the implied cut- 
point on the predictor is 
then applied, the result 
is not a sharp break in 



the criterion distribution, because of the errors in 
prediction, as shown in Figure 2. 

The choice of elements for the standards 
then applies to the criterion, not to the tests or other 
predictors. There is usually only one criterion. When 
there are several criteria, one criterion is usually pri- 
mary. In college entrance that is academic success. 
For example, at Johns Hopkins, being a good lacrosse 
player counts for a lot, but the applicant still must 
be able to avoid academic probation. 

Sometimes, success is predicted from a com- 
bination of several test scores and other indicators, 
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and there is no reasonable way to use the Angoff 
method on a regression composite. 

Once the cut-point is set on the criterion, say 
freshman g.p.a., then it is possible to determine what 
value of the [composite] predictor yields the requi- 
site probability of success. But then, no cut points 
are implied on the individual predictors, since in a 
regression, the predictors are combined in a compen- 
satory way. 

One example of a regression composite can 
be found in the process the National Collegiate 
Athletic Association (NCAA) uses to set a minimum 



standard for participation in college athletics (NCES, 
1995). This year, the standard for eligibility for par- 
ticipation in college athletics programs applies not 
only to the entrance tests but also to the high school 
g.p.a. Moreover, the description of the decision rule 
is compensatory: if you have lower test scores, you 
need a higher g.p.a.; if you have a lower g.p.a., you 
need higher test scores. The standard can be repre- 
sented as a line in the graph of the predictors, and can 
also be represented by a table, as shown in Table 1. 

For the past decade, I have been working 
on various personnel testing projects for the U.S. 

armed forces. In the U. S. 
Military, applicants 
must pass a cut point on 
the Armed Forces Quali- 
fication Test (AFQT) to 
qualify for entrance. 
Then for each potential 
military job, there is 
a second, job-specific 
standard on some com- 
bination of the 10 tests 
in the Armed Services 
Vocational Aptitude 
Battery (ASVAB). Per- 
sons who can just barely 
qualify for entrance of- 
ten can qualify for only 
a few of the hundreds of 
military specialties. 

Between 1982 
and 1992, the U. S. Mili- 
tary conducted a long 
project to establish stan- 
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dards based on actual hands-on performance (Wigdor 
& Green, 1991). Rather than attending to how well 
the incumbents had done in training school, the stan- 
dards were based on how well they could do their 
job. Moreover, in some cases it was possible to get 
judgments from supervisors of the value of various 
performance levels. In some instances, poor perfor- 
mance while not valuable, is also not threatening, 
whereas in other jobs, poor performance can be dan- 
gerous to the incumbent; in some jobs, poor perfor- 
mance can pose a serious danger to others. Poor per- 
formance by a cook may result in disgruntled col- 
leagues; poor performance by an air traffic control- 
ler may result in dead 
colleagues. 

As one looks at 
the problem of recruiting 
service personnel and 
placing them in jobs, the 
issues of performance 
value and performance 
cost must be weighed. 

Not only are highly tal- 
ented individuals in short 
supply, but they are in 
demand. Civilian compa- 
nies want to hire them 
too. Most of them would 
go to college if they could 
afford to, so the Army 
has instituted a signing 
bonus: in return for a full 
tour of duty, the Army 
will provide money for 
college tuition. This is an 



expensive program, and there have been attempts 
to consider just how much good performance the 
military can afford. There are costs of various sorts 
associated with both good and poor performance. 
Obtaining good performance means paying bonuses 
for signing up; poor performance can be dangerous, 
but what mounts up is the cost of recruiting some- 
one who can't make it through specialty school, or 
who resigns, or is fired before they contribute much 
to the organization. 

The U. S. Military now has an econometric 
model that computes the costs involved in setting 
particular standards (Green & Mavor, 1994). But 



Table 1 



Minimum Standards for Eligibility for Participation in Division I College 
Athletics (abridged from NCES, 1995) 



Core GPA* 


SAT 


or 


ACT 


2.5 or Above 


700 




17 


2.4 


740 




18 


2.3 


780 




19 


2.2 


820 




20 


2.1 


860 




21 


2.0 


900 




21 



‘Core GPA has a detailed definition in terms of course requirements. 



standards are not the main focus of the model. In a 
large organization like the U. S. Military, many other 
jobs also need recruits. The jobs are in competition 
with each other for the available talent, and talented 
applicants are in short supply. If some jobs got many 
of the recruits with great promise, the other jobs 
would have to settle for second best. In order to try 
to parcel the talent equitably, each job, or each class 
of jobs establishes a goal of expected performance, 
expressed as a distribution. It is openly recognized 
that all new incumbents are not going to develop into 
experts, but there is plenty of work for journeymen. 
Even if most eventually became superior workers, 
there is always movement into the job from new 
hires, and exodus due to promotion and attrition, 
which in the case of Military, includes finishing a 
contracted tour of duty with no interest by the 
employer or the employee in signing a new contract. 
So for all these reasons, as well as the inevitable 
individual differences, the quality of job performance 
among incumbents at any moment is best described 
as a distribution. That distribution becomes 
a goal for the current recruiting effort. If it is 
decided to raise the goal, by raising the mean 
of the expected performance distribution, then there 
will be cost implications. More effort will have to go 
into recruiting. Moreover, raising the goal in one job 
means draining talent from other jobs, unless the 
means of their desired performance distributions are 
also raised. So a system of balancing has to be used. 
The econometric model recognizes the need for bal- 
ance. The model can also address the "down-sizing" 
economic question: How will performance suffer if 
the recruiting budget is lowered? 



Note that our standards for prediction have 
changed into performance goals, and that the goals 
are expressed as distributions. Setting cut points on 
the entrance tests has been done indirectly by set- 
ting performance standards. Each job has an entrance 
standard, but it is of marginal interest. The main 
interest is the whole distribution, which could be 
specified by assigning percentages above various 
points ordered along the performance continuum. 
Some of us tried unsuccessfully to promote the use 
of such points, to distinguish categories of job per- 
formance (novice, apprentice, journeyman, master, 
and expert). 

Standards as Descriptors 

The simplest use of standards is to clarify the 
meaning of a scale. That was the point of suggesting 
the categories of job performance. When a scale is 
being used for some kind of individual or system- 
wide evaluation, the meaning of various points on 
the scale need clarification. An excellent example is 
the NAEP scales, as described by the NAEP profi- 
ciency levels, also known as anchor points, developed 
in the mid 1980's (Beaton & Zwick, 1992). 

NAEP assesses several achievement areas: 
reading, writing, math, science, and occasionally 
some other areas. There is a NAEP scale for each area. 
Each scale is centered at 250, and ranges from 0 
to 500. Some points were selected as anchor points, 
also called proficiency levels, and descriptive phrases 
were developed in order to characterize the skills 
represented by each level. As an example. Table 2 
shows the mathematics proficiency descriptors. These 
descriptors were developed by examining the items 
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that were generally answered correctly by most of 
the persons below the level, and by few of the per- 
sons above that level. 

The NAEP scales are developmental, in the 
sense that one scale is used to describe the progress 
of students through the first 12 years of school. There 
are discussions about whether the scale is 
really measuring the same thing at lower score lev- 
els as it does at the higher levels, but that is not the 
issue today. The point here is that the proficiency 
levels are descriptors. Standards could be set for the 
system by aiming for a distribution of 17 year olds 
as 100% at level 200, 90 % at or above 250, 60% at 
or above 300, and 20% 
at or above 350. These 
would be goals to aspire 
to, not really likely 
to be met soon, without 
incredible change in 
the educational system. 

Nevertheless, they would 
constitute standards for 
the system. It would be 
almost as good to set the 
standard in terms of the 
mean and standard de- 
viation of the proficiency 
distribution, but such a 
specification would not 
be so easily understood 
by the general public. 

Since the levels 
are being used as descrip- 
tors, it doesn't much 
matter which points are 



chosen, nor how many. There should be enough 
points, well-spaced along the scale, but four or six 
would have done about as well. The important meth- 
odological point is that when standards are used as 
descriptors, one can pick the points first, and then 
find appropriate descriptions by examining the item 
information. 

There has been a recent uproar about NAEP 
standards because of a change called for by a new 
National Assessment Governing Board (NAGB). This 
new board was appointed a few years ago, in the 
midst of a national cry for educational improvement. 
Employers moaned that high school graduates 



4 


Table 2 








Percentages of Students Performing at or Above Mathematics 
Proficiency Levels: 1986 




Level 


Description 


Age 9 


Age 13 Age 17 


350 


Can solve multi-step problems and use 
basic algebra 


0.0 


0.4 


6.4 


300 


Can compute with decimals, fractions, 
and percents: recognize geometric 
figures: and solve simple equations; 
and use moderately complex 
reasoning 


0.6 


15.9 


51.1 


250 


Can add, subtract, multiply and 
divide using whole numbers, and 
solve one-step problems 


20.8 


73.1 


96.0 


200 


Can add and subtract two-digit 
numbers and recognize relationships 
among coins 


73.9 


98.5 


99.9 


150 


Knows some basic addition and 
subtraction facts 


97.8 


100.0 


100.0 








standards. Methods of setting performance standards 
assume that the test adequately reflects the domain 
of knowledge on which the test-takers are being 
evaluated. A prior step would seem to be to define 
the domain. 

The performance standards have to reflect 
the content standards. The bridge from the content 
standards to the performance standards depends 
on the test specifications, the item writers and test 
editors, and on the resulting performance measure- 
ment scale. Logically, it would seem preferable for 
the judges to set standards first on the content 
domain. They could identify what parts of the 
domain are basic, what parts go with proficient per- 
sons, and what parts 
would mainly be mas- 
tered by advanced stu- 
dents. It is not at all clear 
how to do this, but a way 
might be found. Judges 
might also be useful 
in evaluating the bridge 
from content to perfor- 
mance. This would seem 
a more straightforward 
task than imagining 
the test behavior of mar- 
ginally competent test- 
takers. 

NAEP has used 
what they called frame- 
works to delineate the 
assessment content. 

Forsyth (1991) criticized 
the NAEP anchor point 



descriptions primarily because the link back to the 
frameworks was unclear. He felt that even if the 
anchor points were adequately described by certain 
items, it was still necessary to show that the items 
adequately represented their framework. The NAEP 
frameworks are cross-classified by subject knowledge 
and levels of understanding. For example, the Read- 
ing framework (Table 3) is a matrix. The number of 
items representing each cell can be counted. The 
framework for the new mathematics assessment is 
more complex, consisting of five broad content 
strands, three mathematical abilities, and three uni- 
fying themes (Figure 3). An item might represent 
more than one aspect of this multi-way frame 





Table 3 


NAEP Reading Proficiency Levels 


350 


Can synthesize and learn from specialized reading materials 


300 


Can find, understand, summarize, and explain relatively 
complicated information 


250 


Can search for specific information, interrelate ideas, and 
make generalizations 


200 


Can comprehend specific or sequentially related material 


150 


Can carry out simple, discrete reading tasks 
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couldn't do much, and scholars interested in cross- 
cultural comparisons pointed out that United States 
students were not Number 1; far from it. So NAGB 
decided to set standards in terms of a new set of 
generic levels on the scales, which were called 
achievement levels, and were named Basic, Proficient, 
and Advanced (NAEP, 1993; Lissitz & Borque, 1995). 
That is, rather than picking the points first, and then 
characterizing them, they picked the names first, and 
then tried to find out where their new labels belonged 
on the scales. That has turned out to have been ill- 
advised. My colleague, Warren Torgerson, says that 
asking someone to place "Proficient" on a test score 
scale is like asking someone draw a "moderately long 
line" on a sheet of paper. There are bound to be dif- 
ferent notions of what the labels mean. Moreover, 
any method that is used to find locations for the 
labels on the scale had better do all the 
labels at once, rather than one at a time, lest profi- 
cient turns out to be placed higher than advanced. 
That actually happened once or twice, according to 
hearsay. Moreover, the implication that Basic in 
mathematics means roughly the same as Basic on 
writing is no more than a suggestion. 



all students. The NAGB did so when it chose to have 
three standards, leading to four regions of the 
achievement dimension: Unsatisfactory, Basic, Pro- 
ficient, and Advanced. Perhaps everyone should be 
expected to reach at least a basic level, but there will 
always be some for whom Basic is in itself a notable 
achievement. Others could easily become proficient, 
and some should be expected to reach expert status 
and beyond. (This seems not very different from the 
classic A, B, C, D). 

In the context of goals, almost any levels would 
serve. The goal for the system could just as well be speci- 
fied in terms of the proportions of students expected to 
be in the various categories. In that case, roughly any 
categories would do — they are only being used as 
descriptors. In particular, the NAGB could have used 
the NAEP anchor points. There was no need to shift to 
another set of points, and no need at all to try to locate 
those arbitrary points empirically. Of course, NAGB 
and NAEP are aiming only at the system level. Some 
states are aiming at the student level — i.e., using an 
assessment that is long enough to provide students 
some feedback about the quality of their individual per- 
formance. At the individual level, accessible goals are 
needed for all students. 



The main reason for setting high standards 
for educational achievement is partly for assessment, 
and partly for motivation. Teachers are being 
encouraged to raise the achievement distribution of 
their students. The standards are goals again. Those 
who set the goals must recognize that some students 
will be better than others. Individual differences are 
inevitable. The standards should provide goals fop 



C o ; i f e i it St iv ul ci rds 

As noted above, there is an important dis- 
tinction between content standards, which define the 
extent of the domain to be tested, and performance 
standards, which indicate how much of the domain 
has been mastered. Messick (1994) has recently 
argued that the bridge from the one to the other is 
of central importance in validating performance 
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The bridge from such content standards to perfor- 
mance standards is complex (Lane, 1993). 

As many of you know, various groups of edu- 
cators have been devising a new round of content 
specifications for what students should know about 
their given area. The mathematics standards have 
been published (National Research Council Math- 
ematics Sciences Education Board, 1993.), and some 
science standards are on the way. These standards 
are not designed to facilitate measurement. They are 
standards for the curriculum, not the assessment. 
Indeed some of these groups do not place a high value 
on tests. The mathematics standards opens with a 



quote from an Iowa farmer, "You can't fatten a hog 
by weighing it." One possible reply would be, "You 
can't tell about the feed without weighing the hog." 
Despite this brilliant repartee, we can be sure that 
the content standards are not stated in a manner that 
leads in any direct way to content specs for tests. 
Moreover, the current wave of content standards ap- 
pear to be focused on expert performance. The goal 
is deep thought, and expert problem solving. Not 
everyone can be an expert. Everyone can be good at 
something, but most of us get along without being 
proficient in everything. 



Figure 3 



Dimensions of the 1994 NAEP Mathematics Assessment 

Content Strands 




SOURCE. U.S. Department of Education, National Center tor Education Statistics, National Assessment 
Washingto B< QC^ Mathematics Framework for the National Assessment of Educational Progress. 
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Conclusion 



n summary, the psychometric problem of deter- 
mining just where a cut-point should be placed on a 
scale seems not to be a central feature of standard 
setting. Cut points are important in certification, but 
so are deciding what to test and how to test it. In 
prediction, placing the standard on the right scale is 
important. For description and for motivation, 
the placement of the points is less important than 
having enough points to serve as descriptions and 
goals for the full range. And finally, finding a way 
to map content standards onto performance stan- 
dards is a challenge. 
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Literacy: Economic Key for the New Millennium 



Recent developments such as 
NAFTA and the Maastricht 
Treaty, which established the 
European market, have 
opened the doors to world 
trade. Globalization is not new, 
but until now shifts in the relative 
comparative advantage of 
nations have occurred slowly 
and gradually. As the new mil- 
lennium approaches, major 
shifts that influence the com- 
petitiveness of nations are likely 
to occur much more quickly. 

Certain countries, firms, and 
individuals are well positioned to 
compete successfully in global 
markets; others may have diffi- 
culty taking advantage of the 
opportunities. A massive reallo- 
cation of labor is expected to 
occur as Organization for Eco- 
nomic Cooperation and Devel- 
opment (OECD) countries try to 
adapt and maintain their eco- 
nomic positions. 

Traditional job and career 
patterns are dissolving as the 
knowledge content of jobs 
evolves and the need for low-skill 
workers fades. As businesses and 
labor markets change, some 
jobs become obsolete and new 
ones, likely requiring more liter- 
ate workers, are created. In this 
environment of change people 
will need to be prepared to 



change jobs, perhaps many 
times. Thus, lifelong learning and 
literacy become key. 

Lifelong learning is an impor- 
tant means of acquiring new 
competencies and qualifica- 
tions, but adequate foundation 
skills are critical. Governments 
can no longer rely on policies of 
expanding school enrollments 
and improving educational 
systems to meet the demands of 
new and high-level competen- 
cies generated by the economy. 
Because literacy has an effect 
on the ability of workers to learn 
efficiently and to be flexible in 
learning, it also has an effect on 
the rate at which a culture of 
lifelong learning can be realized. 
For some, it is the sine qua non of 
workplace learning. 

According to the OECD, the 
best way of exploiting the new 
economic environment is to 
strengthen the capacity of 
businesses and labor markets to 
adjust to change, improve their 
productivity, and capitalize on 
innovation. But these capacities 
depend first and foremost on 
the knowledge and skills of the 
population. People are the key 
resource and their level of lit- 
eracy is a powerful determinant 
of a country's innovative and 
adaptive capacity. 



Cultivating and developing 
literacy should be an important 
element in every country's long- 
term policy strategies. Systematic 
knowledge about the dimension; 
and levels of literacy and sound 
information about its distribution 
in the population are prerequi- 
sites for forming good policy. The 
International Adult Literacy Sur- 
vey (IALS), which is summarized ir 
this issue of ETS Policy Notes, was 
an effort to begin providing such 
information.® 
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Literacy I 

• Literacy: Economic Key for 
the New Millennium 

• Distribution of Literacy and 
Related Factors 
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published by the Organization for Eco- 
nomic Cooperation and Development 
and Statistics Canada. Unless otherwise 
noted, all data are drawn from that report. 
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2001 L Street. NW. Suite 650 
Washington. DC 20036-4910 
Phone - 202-785-6323 , 

Fax - 202-785-0530 . .; 
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The Distribution of 
Literacy and 
Reiated Factors 

The countries involved in the IALS 
differ in their demographic 
makeup, their educational sys- 
tems, and their employment 
opportunities. Thus, it was not the 
purpose of the IALS to rank 
literacy skills across countries. It is 
instructive, however, to explore 
the differences among countries 
with respect to literacy measures 
and to examine factors related 
to literacy skills. These factors 
include those that lead to 
literacy, such as education, and 
those that might be thought of 
as consequences of literacy, 
such as income and occupa- 
tion. This section of the newslet- 
ter describes the distribution of 
literacy — prose, document, and 
quantitative — in the participat- 
ing countries and illustrates the 
relationship between literacy 
and these other factors. 



Results 

Figure 1 shows the distribution 
of literacy in each participating 
country. The graphs present the 
estimated percentage of each 
country's population at each 
level on each of the three 
literacy scales.’ 



Some interesting findings 

include: 

• Canada and the United 
States have quite similar 
distribution patterns, though 
the United States has a 
slightly larger proportion of its 
population at Level 1 . What 
distinguishes both countries is 
that both have relatively 
large percentages of their 
population at Level 1 and at 
Level 4/5. In both countries, 
there are larger proportions 
of the population at Level 1 
on the document scale than 
at Level 1 on the other two 
scales. Likewise, Canada has 
larger percentages of its 
population at Level 4/5 on 
the document scale than on 
the prose and quantitative 
scales, while the United 
States has smaller percent- 
ages at Level 4/5 on the 
document scale than on the 
other two scales. 

• Germany demonstrates 
considerable variation 
across scales, particularly 
when comparing the pro- 
portion of its population at 
Level 4/5 on the prose scale 
with the percentage of the 
population at Level 4/5 on 
the quantitative scale. 

• The Netherlands shows great 
internal consistency across 
scales, with an especially 
large percentage at Level 3 



on all three scales. As in 
Germany, the percentage 
at Level 4/5 on the prose 
scale is smaller than the 
percentage at this level on 
the other two scales. 

• Sweden has substantial pro- 
portions of its population in 
each of the higher levels on 
all three scales. 

• The two language groups 
in Switzerland (French and 
German) show few differ- 
ences. There are substantial 
percentages at Level 3 on all 
three scales, and there is a 
large proportion at the 
higher levels on the quantita- 
tive scale. In this way, Swit- 
zerland is similar to the 
Netherlands and Germany. 



Employment 

In all the participating coun- 
tries, employment is positively 
related to literacy — those 
employed are more likely to be 
at a higher literacy level than 
those who are unemployed. In 
all eight countries, a very small 
percentage of individuals at 
Levels 3 and 4/5 find themselves 
unemployed, while a large 
proportion of those at Level 1 
are without work. The proportion 
at Level 1 who are unemployed 
is comparatively large in Ger- 
many, Sweden, and Poland. 
These data show clearly that 
literacy and employment are 
strongly linked. 



’Literacy is defined in terms of three domains — prose, document, and quantitative. In each 
of the three domains, a scale from 0 to 500 was constructed. For analytical purposes it is useful 
to group people into five levels of literacy, corresponding to ranges of scores achieved (e.g.. 
Level 1 . the lowest literacy level, includes scores from 0 to 225). Because the proportion at 
Level 5 is under 5 percent in most countries. Levels 4 and 5 have been combined in the IALS 
report. 
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Income 




In all the countries there is a clear 
direct effect of literacy on wage 
income. Individuals at Level 1 
are much more likely than those 
at other skill levels to have no 
income. At the same time, 
those at Level 4/5 are more 
likely to be in the high-income 
group. But it is also important to 
point out that individuals per- 
forming at Level 3 are also likely 
to have relatively high incomes. 

Occupation 

There is a consistent and 
expected relationship between 
literacy and occupation, and 
there are also some interesting 
differences across the scales for 
particular occupations. 

• Manager/Professional is 

predominantly a high-skill 
group with 60 to 75 percent 
at Levels 3 and 4/5. This 
group usually does better on 
the document and quantita- 
tive scales, compared to the 
prose scale. 

• Technician is also a high-skill 
occupation, although there 
are more technicians than 
managers and professionals 
at Level 2, and fewer techni- 
cians at Level 4/5. 

• Clerks display higher literacy 
skills than those shown by 
other occupations with simi- 
lar educational qualifica- 
tions, such as sales, service, 
skilled craft workers, and 
machine operators. How- 
ever, the largest proportion 
of clerks are at Levels 2 and 
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3, not at Level 4/5, suggest- 
ing that the highest level of 
literacy skill is not required for 
the kind of tasks clerks regu- 
larly perform. 

• Sales and Service workers 
are probably the most het- 
erogeneous group — there 
are usually some of these 
workers at each literacy . 
level; and in some countries 
there are significant propor- 
tions in Level 4/5. 

• Skilled Craft workers show 
noticeable differences from 
country to country. In the 
United States and Canada, 
between 25 and 30 percent 
are at Level 1 on the docu- 
ment scale, but in Germany 
only 7 percent are at this 
level. Entry into craft occu- 
pations in North America is 
relatively easy and unregu- 
lated, in contrast to Europe, 
where these occupations 
are usually subject to more 
rigorous entry requirements 
and/or certification based 
on demonstrated skill. 

• Machine Operators have skill 
levels similar to those of 
skilled craft workers when the 
latter are relatively low-skilled 
(as in North America). How- 
ever, they have lower skill 
levels when craft work is 
subject to certification (as in 
Germany). In particular, 
machine operators tend to 
have lower skills on the quan- 
titative and document scales 
than on the prose scale. 



• Agricultural and Primary 

occupations have the low- 
est demonstrated literacy 
skills. This is particularly 
noticeable in countries with 
larger agricultural sectors, 
such as Poland. 



Industry 

There is an interesting and 
important relationship between 
literacy and changes in employ- 
ment opportunity within differ- 
ent types of industries. In partici- 
pating countries, those industries 
that have grown in the last 20 
years, such as financial and 
personal services, are those 
where the incumbents have the 
highest average scores. At the 
same time, industries in decline, 
especially agriculture, are char- 
acterized by workers with the 
lowest average literacy skill. The 
IALS data document the growth 
in skill demand in the changing 
industrial economies. 



Reserve Labor Force 

For an economy to grow 
there should be a source of 
skilled workers to fill the jobs in 
the growth industries. Sources 
include students leaving school, 
the unemployed, and those out 
of the workforce who might be 
attracted back in. In every 
country large proportions of this 
group are at Level 1 , and in 
many countries the percentage 
of the reserve force at Level 4/5 
is about the same as the per- 
centage of unemployed at that 
level; Sweden is the notable 




exception. In certain countries, 
a large proportion of the popu- 
lation outside the labor force is 
at Level 3 (Germany and the 
German-speaking group in 
Switzerland have over 40 per- 
cent at this level), suggesting 
that these countries do have a 
resource that might be avail- 
able if conditions were right. 
Canada, the United States, and 
Poland would seem to have the 
smallest reserve resource; in all 
three countries, over 30 percent 
of this group is at Level 1 . 



Full-time Versus Part-time 
Work 

Part-time workers can be 
a potential source of skilled work- 
ers if they are not less skilled than 
full-time workers. There is no sys- 
tematic relationship between 
literacy skill and whether an 
individual is employed full- or 
part-time. On some scales, in 
some countries, full-time workers 
have slightly larger proportions at 
the higher skill levels; on the other 
scales, often in the same coun- 
try, there are also large propor- 
tions of part-time workers at the 
higher skill levels. There are two 
possible reasons for this. First, 
many part-time workers may be 
students. Second, child care or 
other family responsibilities may 
limit working hours. 



Adult and Continuing 
Education 

The data point to considerable 
similarity between countries both 
in terms of the proportion of the 
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workforce that receives training 
and in the way training is distrib- 
uted by literacy skill. Poland is a 
clear exception — the propor- 
tion reporting having received 
any training is notably small. With 
the exception of Poland, the 
more skilled a person is, the more 
likely he or she has had some 
training. The majority of those at 
Level 4/5 had some training 
opportunity; only in Sweden did 
those at Level 1 have similar 
training opportunities. 

Immigration 

With the exception of 
Canada, immigrants account 
for higher proportions at Level 1 
and lower proportions at Level 
4/5. In Switzerland and the 
United States, immigrants are 
more likely to have lower edu- 
cational attainment than those 
born in the country, but propor- 
tions with low levels of educa- 
tion are similar for both native- 
born and foreign-born residents 
in Germany, the Netherlands, 
and Sweden. At the same time, 
there is little difference in any of 
these countries between immi- 
grants and native-born residents 
in the proportion with tertiary 
education. In these cases, other 
explanations, such as language 
or culture, are necessary. 

Education 

Because the IALS countries 
have quite different patterns of 
educational attainment, it 
should be expected that these 
differences play some role in the 




differences in literacy, since 
there is a strong relationship 
between education and 
literacy. But education is not, by 
itself, a satisfactory or simple 
proxy for literacy. Not only does 
every country have some pro- 
portion of its least-educated 
population at Level 3 and Level 
4/5 on each literacy scale (and 
some of its most educated at 
Level 1), but the relationship 
between education and literacy 
is not the same in every country 
and not the same from scale to 
scale. Thus, comparisons based 
solely on educational attain- 
ment may incorrectly estimate 
true skill differences. 



Parent Education 

While there is a relationship 
between parent education and 
tested literacy, that relationship 
is not the same in all cases. For 
..example, more Canadians than 
Germans whose parents have 
relatively low levels of educa- 
tion nonetheless attain Level 
4/5. It is also worthwhile noting 
that the IALS countries are quite 
varied in parents' education. 



Age 

Education is not only distributed 
differently among the countries, 
but it is also distributed differ- 
ently by age within each coun- 
try. Literacy is related to age 
independently of education as 
well. The relationship between 
age and literacy is slightly more 
complex than the relationship 
between education and 
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literacy. In all countries except 
the United States, at older ages, 
the proportion at Level 1 is larger 
on all three scales. It is generally 
the case that fewer — in many 
cases significantly fewer — 
young adults are at the lowest 
level on any scale. This does not 
mean, however, that propor- 
tionately more young adults are 
at Level 4/5. The mixed results 
seen in these data demonstrate 
that adult literacy is a result not 
just of school experience, but 
also of life experience. This 
implies that the IALS results are 
not an appropriate measure of 
school effectiveness, but rather 
a measure of the culture of 
literacy in a particular society. It 
is safe to say that in all countries 
the proportion of young adults 
who will be entering the work- 
force, on average, have notably 
higher skill levels than older work- 
ers who will be retiring. 



Gender 

The data show that there is 
a gender effect, small in some 
countries, large in others, but 
that effect is not the same on 
all the scales or for all countries. 
In general, as one moves across 
prose to document to quantita- 
tive scales, men's scores 
increase relative to women's. In 
some countries, women's scores 
decrease through this progres- 
sion of scales, but even when 
they do increase they do so 
more slowly than those of men. 
The predominant explanation 
for these differences points to 
different patterns of course 
enrollments in school.s 
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The Practices of 
Literacy 

The IALS collected a broad 
range of information about the 
literacy practices and other 
daily activities of the respon- 
dents and provided new insights 
into the differences in the distri- 
bution of literacy within and 
among participating countries. 
This article discusses literacy 
practices at work and in the 
community. Bar charts show 
the overall frequency for each 
literacy activity across the par- 
ticipating countries. Within 
each section, the relationship 
between literacy level and both 
literacy activity and occupation 
is briefly discussed. 

In general, there are differ- 
ences among the countries in 
the frequencies reported for the 
different literacy tasks, but these 
differences are mostly consis- 
tent from task to task. Swedish 
respondents almost always 
reported the most frequent use 
of literacy tasks at work and 
Polish respondents the least 
frequent. 

These differences reflect the 
countries' different occupa- 
tional distributions. In general, 
respondents in Poland reported 
using all the literacy skills the 
least, and this country — at least 
among the survey countries — 
has the largest proportion of 
workers in the occupations 
requiring the fewest literacy 
skills: agriculture and other pri- 
mary occupations. At the same 
time, Poland recorded the 
smallest proportions in the 
occupations requiring the most 




Figure 2. Proportion of Population in Each Country Who Reported 
Engaging in Various Workplace Reading Tasks at Least Once a Week 
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Figure 3. Proportion of Population in Each Country Who Reported 
Engaging in Each of Several Workplace Writing Tasks at Least Once 
a Week :• .Zf-i'?-- 



Letters or memos 



Forms or items such as bills, 
invoices or budgeted- 





Estimates or technical 
specifications - 




tequent use of literacy: manag- 
srs, technicians, and clerks. 

Literacy at Work 

Reading. Respondents were 
Dsked how frequently they read 
Dr used information from six types 
of texts as part of their job — 
directions or instructions for medi- 
Dines, recipes, and other prod- 
ucts; bills, invoices, spread- 
sheets, or budget tables; dia- 
grams or schematics; manuals or 
reference books, including cata- 
ogs; reports, articles, magazines. 
Dr journals; and letters or memos. 
The proportion of the population 

P each participating country 
ho reported engaging in these 
workplace reading tasks at least 
pnce a week is shown in Figure 2. 

| There is a general tendency, 
across countries, scales, and 
asks, for individuals at higher 
iteracy skill levels to report that 
hey carry out a practice more 
fequently. For example, in 
D oland only 1 8 percent of those 
Dt Level 1 on the document 
scale reported reading directions 
Dt least once a week, but 46 
Dercent of those at Level 4/5 
reported doing so. The differ- 
ences are even larger for tasks 
that are likely to involve more 
complex texts, such as using 
manuals and reference books. 

The frequency of reading tasks 
varies by occupation, as should 
be expected. In French-speaking 
Switzerland, for example, 83 
percent of the clerks reported 
reading memos and letters at 
least once a week, but only 54 
percent of the skilled craft work- 
ers said they read those materials 



that frequently. Also in French- 
speaking Switzerland, 68 percent 
of clerks reported working with 
bills or invoices weekly, while 55 
percent of the technicians said 
that they did. In contrast, 42 
percent of the technicians, com- 
pared with 23 percent of the 
clerks, reported using schematics 
and diagrams this often. Overall, 
the occupational category with 
the highest reported frequencies 
across tasks is the professional/ 
management group. Clerks and 
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technicians reported the second 
most frequent use for many tasks. 

Writing. Respondents were 
asked how often they wrote or 
filled out four types of text as 
part of their job — letters or 
memos; forms or items such as 
bills, invoices, or budgets; 
reports or articles; and estimates 
or technical specifications. Over- 
all frequencies for each country 
are shown in Figure 3. 
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Figure 4. Proportion of Population in Each Country Who Reported Engag- 
ing in Each of Two Workplace Numeracy Tasks at Least Once a Week 



Measure or estimate the 
v;r; ' size and weight of objects 



Calculate prices, costs, 
or budgets 





In some countries there is little 
difference reported among 
those at Levels 2, 3, and 4/5 on 
writing estimates and on work- 
ing on bills and invoices. In con- 
trast, in all surveyed countries 
respondents at Level 4/5 are 
considerably more likely than 
those at Level 2 to write letters 
and memos; they are also 
somewhat more likely to write 
reports and articles. As the task 
becomes more complex there is 
a stronger relationship between 
frequency and literacy level. 

As was the case with the 
reading tasks, there are differ- 
ences among occupations in 
writing frequency. Professionals 
and managers, technicians, 
and clerks reported the most 
frequent use of a variety of 
tasks. Notably, clerks are more 
likely than any of the other 
occupations to process bills or 
invoices; and service workers 
reported as much use of these 
kinds of tasks as professionals/ 
managers and technicians. 

Mathematics. I ALS respon- 
dents were asked to report 
how often they engaged in 
two workplace numeracy tasks 
— measuring or estimating the 
size and weight of objects and 
calculating prices, costs, or 
budgets. The overall frequen- 
cies for each country are shown 
in Figure 4. 

There are some interesting 
correlational differences 
between these two activities. 
Level 1 respondents reported 



using measurement math more 
often than budget math, though 
the overall frequencies of the 
two types of math are similar in 
roost of the countries. Corre- 
spondingly, those at Level 4/5 
reported using budget math 
more frequently than measure- 
ment math. The proportions for 
the two sets of math tasks are 
almost identical in both Level 2 
and Level 3, although higher 
frequencies were uniformly 
reported for Level 3, compared 
with Level 2. 

Clerks and service workers, as 
might be expected, reported 
using budget math more fre- 
quently. On the other hand, 
skilled craft workers, machine 
operators, and agricultural work- 



ers reported using measurement 
math more frequently. Techni- 
cians and professionals and i 
managers reported similar fre- I 
quencies for the two types of 
math. 



Literacy in the Community 

Literacy activities. Much lit- 
eracy activity takes place out- 
side the workplace. The IALS 
asked respondents a variety of 
questions about their everyday 
literacy practices and their 
participation in other social and 
community activities — news 
paper reading, book and 
magazine reading, letter writing, 
and library use. The overall fre- 
quencies are shown in Figure 5’. 



'The overall frequency data are from unpublished compufer runs conducted by 
Educational Testing Service. 
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Figure 5. Proportion of Population in Each Country Who Reported 
Engaging in Various Literacy Activities /•’ 



Pe rce ntage who read a ; he wspape r 
or magazine' every day!: 




Percentage who write a 
letter at least once a week 



v Swtaerland (French) 
Switzerland (German) 



Germany 
: Canada H 



Netherlands 
>' Poland J 

•• •-Sweden I 



Percentage who read books 
at least once a week 




Percentage who use a public 
library at least once a month 
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Daily newspaper or magazine 
reading is fairly common in 
all of the surveyed countries, 
although less so in countries with 
large numbers of second-lan- 
guage speakers (Canada, the 
United States, and Switzerland). 
Literacy level has only a small 
effect on newspaper and 
magazine reading, probably 
because this task covers a 
broad range of skills. 



Less common are literacy 
activities that might be thought 
of as less functional, or "more 
literate," than reading newspa- 
pers. Fifty-five percent or less of 
the respondents in all participat- 
ing countries reported reading 
books at least once a week. 
Reading books has a strong 
effect on the level of literacy. 
Uniformly, more of those at 
Level 4/5 reported reading 



books at least once a week, 
with smaller numbers at each 
lower literacy level. 

There are small differences in 
letter writing across the partici- 
pating countries, with the excep- 
tions of Poland and Sweden, 
who lag behind. At the same 
time, there are significant differ- 
ences across literacy levels for 
all frequencies of letter writing. 

One-third or fewer of each 
country's respondents said that 
they visited a library at least 
once a month. The IALS data 
suggest that the libraries are 
serving the segment of the 
population who are already 
highly skilled. In all countries, 
individuals at Level 4/5 were 
most likely to report frequent 
library visits, although in the 
Netherlands those at Level 3 
reported using the library at 
almost the same frequency as 
those at Level 4/5. 

Overall, adults in those coun- 
tries where the average scores 
are the highest (notably Swe- 
den and the Netherlands) also 
reported the greatest use of 
literacy-related tasks, and those 
in countries with the lowest 
scores reported the least use. 

Activities related to literacy. 

The IALS examined a number 
of other practices that, while 
not direct evidence of literacy, 
were widely thought to be 
associated with it, or with its 
absence. Figure 6 shows the 
overall frequencies for television 
watching and community 
participation. 2 



^The data on overall frequencies are from unpublished computer runs conducted by 
Educational Testing Service. 
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The data provide support for 
the widespread public belief 
that television watching and 
literacy are somehow incom- 
patible, although the relation- 
ship is complex. The iALS data 
demonstrate a noticeable — 
and negative — link between 
the two. Those most likely to 
watch television for significant 
periods of time are usually at 
lower literacy levels. About half 
of the respondents in Germany 
and the Netherlands reported 
watching television more than 
two hours per day. 

There are also considerable 
differences in community 
participation across countries. 
Almost half of Sweden's respon- 
dents reported participating in 
a community organization at 
least once a month, compared 
to only 9 percent in Poland. 
While there are some country- 
to-country differences in how 
literacy relates to community 
participation, in all cases indi- 
viduals in Level 1 are less likely 
to participate in community 
organizations.® 



Figure 6. Proportion of Population in Each Country Who Reported 
Engaging in Various Literacy Activities % 
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onclusions 



The IALS has broken new 
bround in the understanding of 
literacy, its distribution, and its 
implications. When the project 
ivas conceived in 1991 , there 
Jvere reservations about the 
potential for comparing literacy 
proficiency across languages 
pnd cultures. In practice, the 
richness and validity of the data 
pbtained exceeded even the 
expectations of the project's 
most enthusiastic supporters. 

■ The information provided here 
pnd in the IALS report have 
made only a start in interpreting 
the IALS data. The main work of 
Interpretation and application 
will need to be carried out 
within each of the participating 
countries. Some conclusions 
ban already be drawn. The 
most important findings follow. 

i Important differences in 
literacy skills do exist across 
and within nations. These 
differences are large 
enough to matter both 
socially and economically. 
They concern not only the 
overall levels of literacy skill 
in particular countries, but 
also the distribution of those 
levels. In some IALS coun- 
tries, most of the population 
clusters into a relatively nar- 
row band of proficiency; in 
others, there is a wide range 
of difference between 
adults with low and high 
levels of literacy. 



• Literacy skill deficits are 
found not just among 
marginalized groups, but 
affect large proportions of 
the entire adult population. 
Over half of adults in some 
countries fail to move 
beyond the two basic levels 
(Levels 1 and 2) of literacy. 
There is a need to consider 
methods for improving the 
skills of entire populations as 
well as seeking remedial 
measures for selected 
groups. 

• Literacy is strongly associ- 
ated with life chances and 
use of opportunities While 
the processes that lead to 
this result are certainly com- 
plex, there can be no doubt 
about its importance to 
employment stability, the 
incidence of unemploy- 
ment, and income. More- 
over, in most countries the 

. structural adjustment that is 
reducing the economic 
prospects of adults with low 
literacy skills is far from com- 
plete. Therefore, those with 
low literacy levels will have 
even fewer opportunities in 
the future. 

• Literacy is not synonymous 
with educational attainment. 

Not surprisingly, people with 
more education tend, on 
average, to have higher 
literacy levels. But the length 
of initial schooling and fur- 
ther education is shown to 
be only one factor contribut- 



ing to literacy in adulthood 
— in two ways. First, in every 
country there are many 
cases of poorly educated 
people who perform well on 
the literacy scales, and a 
smaller but still significant 
number of highly educated 
people who perform poorly 
on the literacy scales. The 
implication is that although 
formal education yields an 
immense advantage, it is 
also possible for individuals 
to improve their literacy 
through their own efforts and 
behavior. Second, there are 
considerable differences 
between countries in terms 
of the likelihood that some- 
one with a particular quan- 
tity of education will perform 
at a particular level. The 
implication is that schooling 
provides no more than a 
"start in life" when it comes 
to acquiring literacy skills, 
and it appears to provide a 
more effective start in some 
countries than in others. 

• Literacy skills, like muscles, 
are maintained and 
strengthened through regu- 
lar use. Formal education 
systems provide only the raw 
material for adult literacy. 
The evidence shows that 
the lack of application of 
literacy in daily activities is 
associated with lower levels 
of performance. Some sup- 
portive contexts at home 
and at work seem to 
reinforce literacy practices 
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and applications better than 
others. One reason for opti- 
mism is that some areas of 
high employment growth in 
OECD countries, such as 
financial and other service 
sectors, tend to create envi- 
ronments that reinforce 
literacy. Others, such as 
manufacturing, may 
become better at doing so 
through restructuring. But the 
transition to information- 
based economies is not a 
smooth one, and a strong 



effort will be needed to 
ensure that literacy prac- 
tices are improved within 
organizations of all kinds. 

The creation of environments 
that favor lifelong learning 
will require strong commit- 
ment from individuals, em- 
ployers, and governments. 

Adults with low literacy 
levels do not usually 
acknowledge or recognize 
they have a problem. Survey 
participants at all literacy 



levels, when asked whether 
their reading skills were suffi- 
cient to meet their everyday 
needs, replied overwhelm- 
ingly that they were. For 
those designing programs to 
reduce the dependence of 
individuals with low literacy 
levels on others, this denial 
has important implications, a 



ETS Policy Notes is published by 
fy the ETS Policy Information Center, 
Educational Testing Service, :iv v. 

^ Princeton, NJ 08541-0001 5' ?: -r 

(609) 734-5694^ 

Internet: pic@ets.org 

/: http://www.ets.org 

Director. ETS Policy Information 
V. Center: Paul E. Barton 

Editor: Richard 1 Coley 

Copyright © 1996 by Educational 
Testing Service. All rights reserved. 

: Educational Testing Service is 
an Affirmative Action/Equal 
Opportunity Employer.- 

Educational Testing Service, ETS, 
and ® are registered trademarks of 
Educational Testing Service. 



NON-PROFIT 
ORGANIZATION 
ETS ' 

US POSTAGE 
PAID 



04202-02536 • Y66M6 • 024829 • Printed in the U.S.A. 



3 

0 












— - . 



0 o 



0 *^Wfe>. 3 °J£b'^ 

's o <? S ^p> 

’ ^4? S? & ^C A 

r ?^ a ® ^ .^rlf.o’ ^ 

6 >°« 



*5 6 \ 



cP 



3 $ 




\*£t 
' S 3 ' 

"• V V .i'g."- 



uw> /rk r~r2> Yr^ ^oTgl 




< 5 f ^ 



% % 



'*’"*- i r> 



PfE£.®£f° Vi? “P> 
V &'v 3 




as. DEPARTMENT OF EDUCATION 

Office of Educational Research and Improvement (OERI) 
Educational Resources Information Center (ERIC) 



ERIC I 

7 



NOTICE 

REPRODUCTION BASIS 



This document is covered by a signed “Reproduction Release 
(Blanket)” form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release 
form (either “Specific Document” or “Blanket*^. 



ERjt 



